Unix
I have dove deeper into Unix on a number of fronts. I have always been a Centos man, and recently discovered new ways of using Unix. First, I have learned Vim in recent months, and the experience has been very positive. I love being able to fly though my servers, editing code quickly and jumping to the next. I feel a lot of power using it, and I have officially given up on the IDE. I have built myself a nice vim setup and have it all linked up to Github so I can set it up quickly. Here's a nice screenshot
I have also just upped my terminal skills in general. I am really getting a hang of the whole Unix piping idea, and I am having a lot of fun with it. I have been writing some basic Python system level scripts for map/reduce. I am feeling that I may like Unix a lot more than the JVM. I feel free on Unix, and I am really beginning to understand how useful the limitless customization is.
Vagrant Up
As much as I was beginning to really like Unix, I was also running into a lot of issues. I needed different environments for different things, and it was a real pain to set up each time. A buddy of mine turned me on to Vagrant, and I quickly fell in love. Being able to quickly whip up VM's has been a godsend. I now have several boxes that are already set up with all the basics I need. I have functional environment in Centos6, Centos7, and Ubuntu14.04. Each one is set up with Vim and a custom terminal so I can quickly tell them apart. I had been running into the issue of working on the wrong server, because I had lost track. This allows me to easily separate them, they all have custom colors and log on messages. Here's an example:
This has spawned me to create a new repository on Github in which I am just making cloud machines. I call it Machimo, and the basic premise is that I break my development cycle into several machines that can be reused in different apps. So, I have a machine that just does API and exposing things to the web. A machine that just does Data Science with all the libraries preconfigured, and system optimized. A machine that just persists data to databases, and a machine that just runs the databases. Finally, A machine that just scrapes for data using proxies. I am really taking to this idea and I think it's a lot more manageable. I am frequently jumping between working on my own app and trying out things with friends, so it's nice to be able to reuse these things within a component framework. Here is the link to the github.
Docker
So, if you know Vagrant, you now at least know of Docker. It's containers are just VM's but lighter. I have been plain around with these as well and we have a bit of a love/hate relationship. On one hand they are wonderfully light, which makes them much easier to share than the Vagrant package route. The ability to pull then down in production is really nice, and being able to just replicate instances so quickly makes my imagination spin. However, there are a couple drawbacks. First, I'm not sure that it's ready yet. I do know some people using it in production now, and they have expressed some concerns to me. Also, it seems that the documentation isn't really up to date. I had a little trouble getting it going, the official docs left quite a bit to be desired, and it seems they have changed their libraries substantially over versions so much of what you find on Stack Exchange isn't really valid.
I am excited about this technology and think it has a lot of potential. However, I am at a place right now where I'm not wanting to wrestle with low level stuff. I just got Vagrant the way I like it and it works well enough for me. I am more focused on Data Science right now and just don't have as big a need for it. I can see myself using it in the future, but I think I'll wait a year or so before jumping back in.
Python! Data Science!
After spending a couple years learning the basics of development I was hungry for something more. I began to understand how things interacted and became proficient in building applications. However, I came to realize that the questions in my head required I give deeper meaning to the data that I was passing around. I wanted to model things in multiple dimensions. I wanted to quantify the movement of ideas and interactions in cultures. So, I began on my path towards data science.
There are a couple languages you can go with here. I know Scala already and it is a decent choice for big data, but Python and R stick out the most. R seems to be more focused on pure analytics and mathematics, whereas Python is general purpose. It has state of the art math and science libraries, but can also be used in almost all other use cases. So, because I am interested in building intelligent applications, Python seemed to be the right choice for me, and I have been very happy with it so far.
What have I seen in Python? Amazing libraries, I mean really amazing, and they all seem to have excellent documentation. Cool features like list comprehensions. While I like Scala's syntax a little more than Python, there are some really neat things in Python like these comprehensions:
Also, IPython is nothing short of amazing. I have fallen in love with this idea. I can prototype so faster than ever before, and I feel like I can reach an understanding of topics and processes quicker by using it. It looks I am going to be sticking with Python for awhile, it seems like the perfect language for this part of my life.
In the data science direction, I have taken some probability and statistics classes recently. Also, I worked through the Data Science in Python O'Reilly book. And now I have been working through the book and online class Coding the Matrix, which teaches linear algebra in Python. I have really been enjoying it. I know I'm heading the right direction, because it just seems like fun doing it.
Till next time
No comments:
Post a Comment