Posted on

Facebook: Moving Fast at Scale

Facebook: Moving Fast at Scale

“Robert Johnson talks about: the need to prepare for horizontal scalability, very short release cycles associated with a streamlined deploying process, and making the entire process faster every day.”

The neatest thing I got from this presentation was how agile Facebook were.  They do daily releases.  To be able to do this, they’ve put control of QA and deployment back to the engineers.  The engineers are responsible for making sure the rollout happens slowly and to keep an eye for post release bugs.

There are still operations people to assist, but they avoid the common scenario where the operations team gets burnt with some bad deploys and then refuses to roll out further updates, which in turn delays the release cycle, time to get feedback for developers, etc.

The presenter claimed that once they got in the habbit of this, the cycle continued to reinforce itself.  Instead of doing a release every 3-4 weeks, 12 a year, and having 12 lessons learnt a year, by doing em dailyish, you get plenty more lessons learnt from each release and the feedback can be used to improve themselves.

Other neat things they talked about were how they scaled – horizontally across their tiers (web server, index/memcache, persistence/mysql).  It seems a common problem of how to find a way to cluster the information in Facebook in any kind of domain centric database or geographic location since FB users are friends all over the world, in different groups, and ‘like’ different things.  The content on your own FB page is dynamic and pulled from sources all over the world.

He also talked about deploying challenges, in particular getting different versions of php and javascript to interact when different versions of FB code were in the wild (different web servers).  Basically by having a repository lib they wrote called GateKeeper that tells the FB code what version it is running on and essentially which code to follow.  Their code looks a bit yuk on the outset – if this version, do this, if that version, do that – but by doing this they bring the problem of inconsistent behaviour due to having different data structures and code for different versions of the software in the wild back into the hands of the developers to manage.  The code doesn’t cry when different versions interact, it knows what to do which means that they can worry about more pressing things.

Not too technical but a great mix of software practice for distributed systems and something that will hopefully get discussed more and more (and hopefully even taught to undergraduates some day)

Posted on

Another great IDEA presentation

My favourite IDEA presenter, Vaclav Pech is talking about code analysis tools in IDEA.  He talks about

  • Code Inspections (live and on demand)
  • Abstract Syntax Tree
  • The structural code inspection – AST context helps understand the Structure search and replace dialog and how you can use its functionality to make an Idea warning with autofix
  • language analysis
  • the annotations like @NotNull, @Pattern in new JSRs and the IntelliJ specific annotations @Language for analysis of strings
  • stacktrace detection
  • dataflow to/from this
  • duplicate detection
  • dependency analysis
Posted on

Asus M51SN LCD Screen Upgrade

I wanted to upgrade the resolution of my laptop screen recently but couldn’t find any resources in the ASUS forums or elsewhere online of anyone who had done it previously.  The other requirement I had was that I wanted to not only go for a piecemeal ‘next resolution up’, I wanted to go all the way WUXGA (1920×1200) if possible.

The tech specs on the ASUS website for the M51SN only stated they supported screens up to WSXGA (1600×1050) but I did come across a laptop reseller p4laptops that was offering the model with an aftermarket WUXGA upgrade.  This got my hopes up that I could support the change.

Unfortunately the laptop LCD reseller websites I listed, only provided panels for the M51SN up to WSXGA if you were lucky.  Some were really bad only offering WXGA panels for this model – I certainly didn’t want that given I had a WXGA+ already.

I ended up contacting a panel vendor, Screen Country based in Canada who after explaining the situation of what I found were quite helpful in finding what they thought would be a compatible WUXGA panel.  It did come with some heavy disclaimers though – you do have to pay for the postage back if it doesn’t work and it has to be in new condition to return.  After telling them my existing panel number, which for my M51SN-AS037G, was an AUO2277 (B154PW02 V2), they advised me of a Toshiba panel (LTD154EZ0L), I soon put the order in and it took about a week to arive.

The first thing you should do before installing is unplug the power from the laptop and remove the battery.

As for pulling apart the laptop, I found the M50 disassembly guide.  The laptops are pretty close, the only difference is that they have 8 screws surrounding the screen.  The 4 pictures they had regarding LCD disassembly let me know what to expect.  The bits to know here was that I didn’t need to pull off one of the plastic hinges at the bottom of the screen.  It wont matter too much if they stay on during the LCD replacement.

Once the plastic surrounding the screen was unscrewed and snapped off, the existing LCD needed to be removed.  The other useful resource was the Screen Country support docs which explained how to disconnect the existing panel.  They do the disassembly on 2 laptops.  For this, the 2nd diagram more closely fit the Asus’ layout.  The key bit here is not to remove the immediate 4 screws that are on the edges connecting the LCD bracket to the laptop casing.  All you need to worry about are the smaller screws on the left and right side of the monitor connecting the casing to the monitor.

When pulling off the main video cable that runs along the back of the monitory, just take the tape off and use your fingernails and slowly edge out the video connector.  To remove the inverter cable at the bottom (white and pink coloured wires), its probably best to approach it from the right side and pull out the connector.  The inverter connector only snaps in one way too if u forget which way it is.

What tripped me up was putting the new panel in and plugging the video connector back into the laptop.  Its a very small bit of connectivity, and I didn’t push it in enough.  Its a bit counter intuitive but you have to apply a bit of force here.  Its probably best that once you line up the connector, you use the sticky tape thats goes over the connection to help push the connector in.  You’ll know the difference between partly in, and fully in when the connector slides in that couple of mm more.

Once that was sorted out though, I powered on the laptop and could see my windows boot up in new resolution glory.  A nice moment indeed.  Facebook, web browsing and IDE use are the apps that benefit the most from this change.

Posted on

My Interpretation of the Signals and Slots Pattern

BasicWiring

What happens

An ObservableClass ‘emits‘ signals when something interesting happens.

Examples of interesting things:

a property change in a bean;

a new db connection up;

a window closes

Emitting a signal is something that has to be done manually by the programmer after they’ve done the interested thing

Example

setProperty(Object newValue) {

this.property = newValue;

emit(newValue);

}

It means tell all the InterestedClasses to check out the signal and optional newValues passed and do something with it in a slot method

To let the ObservableClass know which InterestedClass methods to call (slots) when we emit the signal, we must connect them to each other.

We could also connect multiple slots to signals

Once connected however, the connection is quite loose.

The slot method is usually referred to as a string (though we could use closures in future). Thus slot or slots to execute when a signal is emitted is determined at runtime, by looking at the parameters in the emit and matching them to the parameters of the connected slots (methods).

Implementation Use Case:

  1. We are interested in one or more discrete events that happens in the lifetime of the Observable Class.
  2. We create Signal objects to represent each of these events that could ever be emitted by the ObservableClass. We usually do this in the constructor of the ObservableClass
  3. We update our methods to emit signals (call the emit method straight after the interesting thing has occured)
  4. If we want to send some event context data to any interested listeners (slots), we can pass parameters to the emit to pass on when the Signal is fired.
  5. In another class (or maybe even same one?) we write a method to execute when the interesting thing happens. This is called a slot
  6. We connect the signal to the slot via a connect method
  7. When the interesting thing happens, the emit method on the Signal object is called. emit is responsible for letting all the connected slot methods to know to fire.

    Other Notes (extentions)

  • A signal can be connected to multiple slots (methods) in any number of different classes
  • You can chain signals together
  • Communication is one way from the Signal event to the InterestedClass.
  • The params in emited in the Signal can be more than the slot method (but not the other way around)

Comparison to the usual way done in Java Swing (Observer Pattern)

In Java, an Observable Event occurs. InterestedClasses must register their interest with the observable class.

The ObservableClass must keep track of who is listening and notify each of them when the event occurs.

The InterestedClasses must all implement the same notification method (and full method signature) in order to use the event. Potentially less runtime mistakes because the IDE will pick up at compile time

Remember the Signals and Slots pattern doesn’t care about the method signatures of who is listening and effectively can fire updates to different methods of different types.

Posted on

Design your own DSL with Groovy

http://www.infoq.com/presentations/Design-Your-Own-DSL-with-Groovy

This presentation is by Guillaume Laforge, one of the Groovy founders, who does quite an interesting presentation about how Groovy supports Domain Specific Languages.

In short he starts with a simple example of how the Groovyisms in the language such as method calls without the need for parenthesis, named params and maps and list param injection into method arguments make for readable DSLs, all without touching any of the fancy meta-programming functions provided.  Very impressive. Other Groovy features that we sometimes forget about are ranges and operator overloading which were demonstrated here and  improve readability.

Then he scratches the surface a little deeper showing an example of how to add methods to existing classes using the metaClass property.  He takes the Number class and dynamically adds a method that constructs an object representing a measure – 6.pills or 6.euros which call a method Dosage getPills() that you inject into Number.  

   1: Number.metaClass.getPills = {

   2:     return new Dosage( numOfPills: delegate );

   3: }

When combined with the previous readability of maps representing method args and parenthesis less method calls, it can lead to some readable code

   1:  take  1.pills

   2:   of: SomeDrug,

   3: after: 6.hours // Number.metaClass.getHours = { … }

take is defined as 

   1: def take(Map m, Dosage d)

This is a great intro to metaClass object protocol for those who haven’t come across it in Groovy.

The last part of the presentation talks about Abstract Syntax Tree transformations and how you can inject syntax into the Groovy language using annotations.  He uses the example of the Spock testing framework and how the @Speck annotation injects behaviours that wouldn’t normally be allowed in Groovy or Java, such as method names which are “strings with spaces and question marks?” and other neat stuff.  He also talks about how its important to limit what your DSLs can do in the JVM if you are giving them off to

I think in summary Groovy is a great language for supporting DSLs and the presentation gives a valuable insight into some of the more advanced groovy features, a worthwhile watch.

Posted on

Brain Computer Interface experiment on real humans

This recent study of patients with epilepsy who prior to going into surgery had the opportunity for interesting brain research to be performed.

The opportunity allowed for electrodes to be wired directly onto the brain similiar to previous expirements on primates.  The subjects were asked to perform an action like moving their arm, or sticking out their tongue, and also to imagine the same movement.  They were then given a computer simulation with a cursor on the screen and asked to do a similiar movement by thinking it.

With about 10 minutes of training, patients were able to control the mouse cursor. The brain got progressively better with more practice.

Posted on

Junit’s Theory’s as interprested by Schauderhaft and Groovy

JUnit theories sound promising.  Many a time a developer writes a whole lot of @Tests along the lines of testParameterXzero(), testParameterXone(), testParameterXmaxInt(). The test code may be almost identical apart from the parameters being used in the class/method under test which is redundant and prone to error.

Theories offer a sound alternate, specifying a single test method, with a different set of annotations to the regular @Test defining a field or annotation based ParameterSupplier to inject a series of values into each test.

Schauderhaft’s blog provides a great summary of what theories are and how to write a parameter supplier to supply a series of datapoints in Java. The annotation based ParameterSupplier does look a little verbose but as Schauderhaft points out, they can be reused – annotation based data fixtures – excellent. I liked these two posts because they do a better job at explaining than the release notes. The blog also links to a blog about another Junit4 feature – parameterised tests – that seem to be a precursor to Theories, and a little simpler to setup. Another great source of how to use Theories is actually in the Groovy documentation about Junit integration. Closures make Datapoints a little more concise (or maybe the example is just simpler? hehe) along with the fact that test data (lists,maps) have a groovy conciseness to them to begin with.

Posted on

JavaFX Charts

JavaFX has some pretty charts in the latest 1.2 SDK.  This post has a general overview, including the javafx.scene.chart class hierarchy and a great demo of all the charts with an excellent click transition.

image image  image

1d vs 2d (Pies vs Everything Else)

Well lets go 1d, a pie chart.  The series of data that fits a pie chart is just a pie label and its value

   1: def pieChart = PieChart {

   2:     title: "Sample Pie"

   3:     data: [

   4:         PieChart.Data { label: "Apples" value: 34  }

   5:         PieChart.Data { label: "Oranges" value: 27 }

   6:         PieChart.Data { label: "Bananas" value: 16 }

   7:         PieChart.Data { label: "Grapes" value: 50 }

   8:         PieChart.Data { label: "Cherries" value: 6 }

   9:         PieChart.Data { label: "Raspberry" value: 7 }

  10:     ]

  11: }

Code snippet taken from How do I create charts and graphs? – not much of a tutorial, but a link to some better references later.

The most important thing is that most charts, with the exception of the pie chart, inherit from XYChart. 

This means that while you may think a simple Bar Chart is just a series of Y values, its actually a bunch (series) of X categories with a complimentary data series (or two or three or more) plotted against those category values.  In the below example are 5 x-categories, with three data series (each a different colour)

image

I would think there should be a SimpleBarChart which only takes one data series and has an interface more like a pie chart, I think it would be a fun exercise as a JavaFX student to roll your own, extend chart, perhaps use Mixins of various pie-chart features (if possible)

The concept of two axis lends itself better to line charts where you are plotting anywhere in the XY space that is allowed.

BarChart Gotchas

Its not as automatic as you’d like.  The BarChart wouldn’t display until I defined my XCategories and set my Y limits (valueAxis).  

   1: BarChart {

   2:     categoryAxis: CategoryAxis {

   3:         categories: ["Child", "Teen", "Peaking", "Middle-Aged", "Grand-parenting", "Refined"]

   4:     }

   5:     valueAxis: NumberAxis {

   6:         lowerBound: 0

   7:         upperBound: 80

   8:         tickUnit: 10

   9:         label: "Age"

  10:     }

It would be nice in future to have an auto-resize=true flag which does as much magic as the proportional: true flag in the LinearGradient class.

Setting Data in Bar Charts

Order is important in the series, as you add a datapoint to the series, it gets assigned in a Bar Chart to the first item in the categories above.

Although a category field exists in the Data item, it doesn’t correlate to the category in the X-axis above. 

   1: data: [BarChart.Series {

   2:         name: "The series",

   3:         data: [

   4:             BarChart.Data {category: "Child", value: 10},

   5:             BarChart.Data {category: "Refined", value: 80},

   6:             BarChart.Data {category: "Peaking", value: 25},

   7:             BarChart.Data {category: "Middle-Aged", value: 40}

   8:             BarChart.Data {category: "Grand-parenting", value:60}

   9:             BarChart.Data {category: "Teen", value:16}

  10:         ]}

  11:      ]

image

The data follows the order of the series, the category in the BarChart.Data literal is ignored – likely used for a different purpose :)  Note to self, RTM (or perhaps file a bug, what I read in the API sounds like I’m doing it right?)

BarChartSimple.fx

Binding the data to a variable

The bar charts data and categories can be bound to another variable, and the variable updated by something else.  In the below code I was able to add an onMouseClicked event to the bar chart so that when the series title or one of the existing bars were clicked, an additional category and random value data point where added to the chart.

BarChartBind.fx

Other Intro’s to JavaFX Charts

Creating XYCharts in JavaFX – The continuation of Dean Iverson’s blog about how to do each of the XY Charts, nice use of the for loop function to prepare the chart data.

Animated Bar Charts – Sexy way to dress up the bars with Fills (gradients or different coloured paints for the bars) and KeyFrames to animate them

What’s New In JavaFX 1.2 Technology: JavaFX Charts – The most detailed blog entry I found about charts from Sun/Oracle so far.  It actually went in depth to all chart types quite well and covered feeding in data from an RSS feed for a stock graph application

Paul Bakker

JavaFX Folder Visualiser – Using BackgroundTasks to update a folder pie chart view

JavaFX Charts – A great rundown of all the chart types with their usage

Interactivity with Web Services

So what about interactivity?  Live feeds from RESTful services are done with the HTTPRequest, then invoking a PullParser on it.  The ShoppingMashup is the first simple example of this, but its real simple and doesnt get too much into the how.

The first thing I had to do though was learn how to use Web Services in WS.  Not wanting to go too off topic, there is a HTTPRequest and a PullParser.  The PullParser does the work of reading an input stream from the HTTPRequest and processing a JSON or XML request.  My issue with it just seems to be the way that you access the XPath or JSON data.  I would have liked a more concise interface, too many if START_ARRAY_ELEMENT, if END_VALUE to make it feel testable.  And you can’t use the same uniform interface to read a JSON resource as you do an XML one.  No where near as simple as what you can get through Grails. I’ll elaborate in a future post.

The best examples to help me get started on a JSON feed though was Coffee Shop, in particular JSONPullParser.fx.  Other tutorials for talking to RESTful service that may help: JavaFX RESTful Web Services Invocation (using XML to talk to a home grown feed), pet catalog javafx example application and JavaFX Twitter Client – all referenced from Connect to a web service or RSS feed on the JavaFX Learn page

See TwitterTrend.fx and TwitterTrendClient.fx

My next TODO is to poll a few days worth of twitter trends and graph them.  And since this post has gone on long enough, I’ll defer that to my next one 🙂