jtv - bash-fu for top cpu consuming threads in jvm along with the output from jstack

So if you ever find yourself needing to find out what threads in a JVM are taking up lots of CPU you can run this little bash script.  It is super stupid and ugly but it gets the job done!



https://github.com/asharif/jtv

Skip host key checking with ssh and scope

Just a reminder cause I can't remember syntax for the life of me!

ssh -o StrictHostKeychecking=no -o UserKnownHostsFile=/dev/null

Get number of threads in a *nix process

Just so I don't forget this:

ps -L -o pid= -p  | wc -l

on OSX(also works on linux I believe)

NUM=`ps M  | wc -l | xargs` && expr $NUM - 1

make-android - Android bloated build system be damned!

Lately I've been working on some Android stuff.  I hadn't worked on android in awhile so I downloaded their new android studio instead of the eclipse ADT.....

Ya I know it's in beta, but are you kidding me?  A sample hello world application took like 30 seconds to build with gradle.  And gradle itself is a bloated tool.  As soon as I attempt to type

$ gradle

in terminal it makes my macbook pro sound like it's taking off into space.

So I tried going back to the ant system in ADT. There I ran into problems as the AndroidManifest.xml now lives in a different place. I tried to look and fix the 1500 line xml garbage under

$ANDROID_HOME/tools/ant/build.xml 

and quickly said WTF.

Then I said you know what I am not going to live like this.  Screw you Google, your build tools are bloated abominations I will make my own.  It turns out that building for android without all the abstractions of gradle and ant is WAY faster and to me more elegant.

The native tools 'aapt' and 'dx' helps us here.

$ANDROID_HOME/build-tools/x.x.x/aapt

$ANDROID_HOME/build-tools/x.x.x/dx

So here is a example using good old Makefile. Yes I know I could of used ant to have incremental builds, but I can do the same in make (future post) without having to spawn a new jvm instance!

https://github.com/asharif/make-android

A lot of it as you can see is to support google play services. I can't believe Googles way is to add a project in eclipse! Anyway a gradle project that took 1 minutes to build on my late 2013 Macbook Pro now takes 7 seconds. That and my laptop doesn't turn into hot coal. Also notice how I left out the part about the keys,keystore and signing. I figure that stuff is outside the scope and regular java stuff.

Enjoy!

ref: The CLI refactoring tool

So one of my coworkers tried to bully me into using Xcode for modifying files. I'm no longer a fan of IDEs like I was in years past so I politely declined as I have vim setup with NERDtree and CommandT. The only thing that I still have to use an IDE for is really java. Especially Java code that I didn't write myself and is ultra verbose with pyramids of class hierarchies to write the most trivial programs!  Regardless, this person who shall remain to be unnamed, insisted and pointed to the refactoring option of Xcode and other IDEs as a huge time saver.

At first I got annoyed.  Why are u forcing your style on me I thought to myself!  Then I realized to my own dismay that he was right.  I didn't even bother telling him I could do something kinda sorta similar with some combination of sed, find and grep.

I then decided to do something about it and ref was that something.  Not sure if I like the name but who cares lol.  Check it out @https://github.com/asharif/ref

Here is a sneak peak!

Hello Hive!

New year....new job!

So the next chapter is in ad-tech and big data. Before a few weeks ago I had only heard of hadoop and hive from a couple articles I had read. Wasn't really sure what MapReduce even meant other than

"MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster" - wikipedia

.....

umm right.....

Fast forward to now.  After some more reading and understanding the above statement is much clearer.  Hadoop basically cuts a "job" up and a master node tells a bunch of worker nodes what part of the data to work on and how to work on them.  This is all abstracted out from the programmer however as once Hadoop is configured the programmer just runs his/her job and hadoop handles the distributing.

So what's hive?   Hive is making hadoop SQLesque.  It's an abstraction of hadoop to let programmer brains reuse SQL skills instead of having to write hadoop map reduce classes all day long.

Anyway on to an example!

The source is available @ https://github.com/asharif/hello_hive

You need to install hadoop and hive and have both available in the PATH.

There are only two files here.  hello_hive.q and hello_hive.py.

hello_hive.q

CREATE TABLE words (word STRING ) ROW FORMAT DELIMITED LINES TERMINATED BY '\n' STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '/Users/asharif/development/hello_hive/input' OVERWRITE INTO TABLE words;

add FILE hello_hive.py;

INSERT OVERWRITE TABLE words
SELECT
TRANSFORM (word) USING 'python hello_hive.py'
AS (word)
FROM words;

SELECT * FROM words;

The above hive script creates an imaginary table using the file in the given path (don't forget to change that path!).  It uses the '\n' as the row delimiter so each new line is a new row in the table.

Then the fun part.  A simple python script is added and the python script is used to overwrite the contents of the table.

hello_hive.py

import sys

for line in sys.stdin:
        print "hive row: " + line

As you can see all the python script does is prepend the text "hive row: " to each row.

Now if you run this

hive -f hello_hive.q

you will get the following output:

hive row: i

hive row: am

hive row: the

hive row: best

hive row: i

hive row: am

Time taken: 0.098 seconds, Fetched: 12 row(s)

FizzBuzz

So I started interviewing after about 5 years for a new position. I was surprised how many places required simple algorithms and sophomore/junior year computer science trivia. This skews the interview in favor of new grads as they are not too far removed from these classes and they are use to artificially solving questions without a compiler and on paper. Maybe this is on purpose to filter out the older crowd?

Word to the wise the interview landscape has changed a bit. Nobody cares what you have done or your experience. Anyway us "old" timers (30 somethings lol) must adapt or become real estate agents. What is the best way? Lots of practice and studying. I busted out my "Intro to Algorithms" by Cormen and went to work. Here is my fizzbuzz repo to help fellow coders out.

https://github.com/asharif/fizzbuzz

Now next step. I need to buy Cracking the coding interview http://www.amazon.com/Cracking-Coding-Interview-Fourth-Edition/dp/145157827X and bust out my notebook and try to get the little practice problems right on paper.