Massive Datasets in JavaOne

I’m both proud and excited to bring the presentation Querying Massive Data Sets with Google BigQuery & Java 8 to the JavaOne audience. I’ve already spoken in Melbourne about Google BigQuery, and its matured nicely over the last year growing in feature set, and we’ve been able to take advantage in more novel ways.

This talk presents a state of the union for where BQ is as a Big Data tool, comparing it to other tools such Hadoop and traditional RDBMS. Most interesting is how to take advantage of BigQuery using the provided Java API, then combining it with the Java 8 Streams API to pull in loads of data on scale.

June 22, 2014

What you may not know about Gradle

Last week I was at the Gradle Summit in Santa Clara. There were a few things I learnt about Gradle that were worth mentioning since they expanded what I knew Gradle to be, and are just handy to know about

Support for Docker, Vagrant & Cargo

Gradle has a docker plugin allowing builds to define their own Docker VM’s. This keeps the development environment the same as production which means less risk when deploying to a new environment given things are more ‘same’. Vagrant and Cargo support do the same for VirtualBox VMs and Arquillian based JEE containers. Developer checks out build and has the production environment (equivalent) ready to go, all orchestrated by the Gradle build. Ben Muschko’s Provisioning Virtualized Infrastructure with Gradle talk has more.
Not just for JVM projects anymore

Gradle has had native support for months now, but it’s been adopted quite quickly and used by companies to build native apps. These companies have little to do with building Groovy or Java apps. Prezi use Gradle to build their JavaScript modules and manage their dependencies, and also build their native mobile apps which incorporate some of those JS modules. It’s enlightening to hear the Prezi devs state they have no JVM knowledge but have taken Gradle in new directions with new modules.
Appreciation of the number of Enterprises dealing with Multi Module builds

There are some large companies using Gradle. LinkedIn, Google, Netflix, Prezi were present at the conference and had talks. You quickly get appreciation for the number of developers using the tools, and the challenges that all have in incorporating dependencies from other teams. Gradle is in a valuable position to provide tooling to help these organisations work amongst themselves, get their builds building more often and more consistently.
Appreciation that working on multiple modules at once that depend on each other gets tricky

When working on module A, that depends on module B, Gradle will download its dependencies and recompile A. When working on both modules B and A, do you want to bring in B from the artifact repository or your local one you are currently working on? There is also the chance that the existing version of the dependency will be loaded from the repository instead of using the one that you are currently building. You want to make sure that doesn’t happen. The team at Prezi created Pride, a way to manage multi project builds and specify a fenced off area to indicate the dependencies you care about building yourself versus those in the dependent modules. Their blog, Simpler Management of Modular App Development, goes into more detail about the scenarios you want to care about and their approach to managing this.
Apart from that I gathered a lot of anecdotal evidence about how large enterprises with multiple teams chose to avoid updating a dependency built by another team for fear of it breaking the build or generating bugs. The problem manifests itself in other bugs that are fixed by that dependency but not implemented by all the products the company releases. There doesnt seem to be a consistent way to manage this but in those larger organisations it does fall onto the build masters responsibility for reporting on the quality of the dependencies used in each project and make sure that they are up to date, and work with the project leads to make sure they get updated accordingly. The challenges of integration within the enterprise came up multiple times to make it worth mentioning here.
The number of attendees a Gradle Summit draws

I was quite impressed with the number of attendees. I dont have an official number, (could guess a few hundred to a thousand?) but the fact that a conference can be built solely off the existence of a build tool, and have enough sessions to run 3 separate tracks over two days is impressive. Sure Silicon Valley has some of the largest Enterprises using Gradle (and thus developers) using the tool, but people also traveled from other parts of the US, and internationally from Europe, and even Australia to talk Gradle. The result is a conference the size of YOW Melbourne, with an equal breadth of attendees. Gradle is positioning itself as *the* polyglot build tool and so the breadth of topics can only grow from here.

Apart from the experience of presenting, Gradle Summit opened my eyes to a bigger picture of what Gradle is about, far beyond the JVM realm I thought it to live.

June 2, 2014

Presenting at Gradle Summit

In a little over a week and a half, I’ll be presenting this session at Gradle Summit in Santa Clara

Single Page JavaScript Webapps … A Gradle Story

Most people ask me why we chose to use Gradle over a JavaScript build tool. It was a small circumstance that we took this path. Usually most companies begin their new JS project with Grunt or Gulp. We tried the same path but it was an unfortunate case at the clients site, their firewall that prevented us from using npm to download npm modules. We tried to get around this but since we already were using Gradle and it could get around the firewall, we put it through its paces.

In addition, the pace of JS build tool momentum is a little ridiculous. Every time I visit a JS meetup there is a new tool on the scene for managing builds. I’m not sure where to invest my time in learning a tool if the flavour of the month changes so regularly.

Many Java projects have picked up Gradle. There is low configuration required to get a build up, but you are allowed to declare more as required to customise the build for your world. Since many JVM devs are learning Gradle, and have migrated some of their projects to the new build tool, the barrier for a Java dev to use this tool to build projects on other platforms like JavaScript isn’t too hard.

In this talk we’ll cover getting a simple build up for a simple single page JavaScript web app covering the functions that are available in Gradle for processing JavaScript & CSS resources. We’ll look at some cool features you can get by integrating Jetty so that developers can checkout and run the project in an interactive way. Other tricks and tips covered whilst developing our project will be covered. Finally, grunt and gulp are the two main contenders in the JS build space. We’ll provide a quick overview of how they work and where you’d want to bring these into your build.

If you are in the area and can come to gradle summit, please come along to watch!

May 26, 2014

Asia Pacific Oracle User Group Leaders Summit 2014

Last week I was in Hong Kong for the APAC Oracle User Group Community Leaders Summit. The summit brings together user group leaders that support Oracle’s products to talk new strategies for the community / user group programs and announcements regarding Oracle’s product direction. The summit is held yearly but there are others held in the United States (IOUC) and in Europe.

Luxurious venue, great location, tasty food

Ok, so first things first, this is why you really go to conferences. The conference took place at the Langham Place hotel in Mongkok. The hotel is well connected to the rest of Hong Kong, the MTR rail system is a short walk from the connecting mall. Despite Hong Kong having a much more humid climate than our home town of Melbourne, you could avoid the humidity almost entirely. From air conditioned airport, to taxi, to hotel, you are in A/C for 99% of your travel. The hotel itself is very comfortable to stay in with great restaurants and a decent bar, ‘the backyard’, for a drink. BTW, the Spa, Sauna, Gym and rooftop pool are all first class. And I can highly recommend getting a facial there, it will help your skin look its radiant best for when you do your 4 minute pitch later in the week (that’s my wifes recommendation)

Ok enough of the material things with just one note, it seems that one thing user group leaders have in common is that they love to travel.

Who

Although I had met many user group leads at conferences like JavaOne and YOW, this was my first user group summit and I hadn’t formally met any user group leaders from this part of the world before. I wasn’t quite sure how exactly who would be there and how it would play out.

Attendees were from the community, user group leaders and Oracle ACE experts from Taiwan, Japan, China, Hong Kong, Sri Lanka, India, New Zealand and Australia. Product managers, architects and community relations managers from Oracle also attended and gave talks.

The key distinction to make is that the summit is for all of Oracle’s products including Database, Peoplesoft, JD Edwards and Java.

Every time Oracle make an acquisition they attempt to integrate the existing communities within Oracle’s user group program. You’d think because each product is so different there isn’t much point of having a conference all under the one roof. But after talking to attendees you quickly realise why. What is common amongst us, apart for love of travel, is that we all lead large communities back home and face common challenges and situations. We may even overlap the same regions as another Oracle products user group. Without these kind of events, we wouldn’t know who else is in the user group fold. We could potentially pool resources for conferences or other one off events in future.

Quickly you start to get an appreciation of the beast that is Oracle. It’s not only a database vendor and it’s not just a steward of the JVM platform. Oracle are equally a software vendor that supplies configurable COTS (Commercial off-the-shelf) software, whose users want modern IT products supporting new technologies (BigData, Mobile, Cloud) just like DBA’s and Java programmers do. The difference being rather than provide the low level platform that developers use to build a custom solution, here they are provided a solution already that needs configuring. Governance plays a significant role in these users day to day, but like our user groups, members join to learn more, and find solutions to common problems.

Agenda

Welcome Reception

Networking, networking, networking was the order of the first evening. Oracle put on a welcome reception at ‘the backyard’ of the Langham Place where we got to meet the Oracle community team. Some had been working for Oracle for over 30 years and they all came from different companies acquired by Oracle. It was great to meet them and put faces to names that I only had dealt with at MelbJVM by email and occasionally phone.

In addition I realised I wasn’t the only Aussie, meeting the lead of the Higher Education User group and the Quest Asia Pacific group – aficionados for some random product called JD Edwards (no it ain’t a paper manufacturer but an ERP solution, learnt a lesson already.)

The networking event was a great way to find the other JUG leaders in the group and talk shop. Turns out that we have quite a bit in common. For example, finding speakers and attendees is one thing. Performing deep freudian psychoanalysis on our communities and the communities of other programming languages to see who would in their right mind want to come to our meetup is another. 🙂

Conference Begins Proper

Day 2 and we get down to business with introductions and a keynote. There is a more formal introduction whereby everybody introduces themselves based off a slide we each prepared a few weeks earlier. You can tell everyone here has done some public speaking before and most people spin off a few sentences describing what they do, their likes, family and employer. I’m especially impressed with those who come from countries where English isn’t a first language and present about themselves in English (hats off to them).

The Agenda is a mix of Oracle product, upcoming SaaS abilities in their COTS software touching on PaaS and IaaS offerings, a panel discussion on Internet of Things, and then the conference breaks into two tracks – one for Java, another for Oracle Tech & Apps.

The Java track begins with a discussion of the recent Java 8 tour. India and China have already had their Java 8 roadshows where Oracle PMs and Java Champions have come to visit or at least do remote talks. A number of JUG leads come up and show pictures from their events and talk about their experiences so far. Another 2 Jug leads, the Taiwanese JUG and my JUG MelbJVM already rolled our own Java 8 launch events and discuss these. Getting content has been done by ourselves and our member base. We’ll be having Oracle down for our formal launch event in June to talk Java 8 and provide an Internet of Things workshop however its also good to have Java product managers tell us that in future they are happy to put us in touch with Java champions to do remote presentations in future – Something we will take them up on.

We then break for lunch, more opportunity to network with Oracle employees.

4 minute pitch

Once back from lunch, we see a set of best practice 4 minute pitches. These are lightning talks, strictly enforced time limits by punishment of an embarrassingly loud phone buzzer, about how each user group runs their community or something regarding the Oracle products they work with.

I have a 4 minute pitch to give on ‘analysing your meetup group using meetup.com’. It worked its way up to a 15 minute deep dive at one stage before being shown the delete slide treatment to get it down to within time. It goes ok if a little rushed, but it’s a bit of fun and apart from communicating ideas, it also is a way to give the fellow leaders more insight into who you are and what you are doing.

The Colombo JUG’s 4 minute pitch on Hackathons is the standout 4 minute pitch for me that session. Defining what they are, and what they are not, is advice we’ll be taking home to implement.

Back to the Java track… with a slight detour

We are then shown 55 new things in Java SE 8, which you can find online yourself. Since I had already seen this before I step into the apps track to see a talk on Oracle’s BigData & Business Analytics. The talks are interesting though I find aimed at a different audience, more high level business focused outcomes versus developer nuts and bolts demos. It helps me appreciate the different kinds of audiences and the members the respective user groups have in each room.

Something I hadn’t seen before was the next session on Java EE 7 and beyond. I have to admit to not being much of an EE container user, but the focus on websockets, JSON processing and HTML 5 looks useful. The fact you can get a lightweight web-only profile container makes me consider these APIs for throwing together a quick server side app… yes, I am still talking about JEE here.

The final talk of the day is on Embedded Java. This one is definitely aimed at the JUG leads and giving them a perspective on Oracle’s position. IoT has always been in Java’s DNA since day dot, and well established through JavaME. Oracle have done a lot of focus improving this position as of late, partnering with Freescale Semiconductor and ARM. The key messaging here is that the hardware manufacturers have realised that building purpose built ICs isn’t viable long term solutions. Embedded logic, with changeable rules are the play of the day here, and bringing as much of the decision making to the client end of a solution, perhaps evaluating in some intermediary on-premise controller, before notifying a server side/external service for help. This hasn’t been easily possible when all this is developed for single platform C++ environments.

JavaME already runs in some extremely low hardware intensive environments (think 16kB Java Cards), there are already libraries that have the right level of certification for use in secure applications – eg SATSA Security and Trust services JSR177. Because this certification is already there it can be used in a variety of government applications without having to be done from scratch. There is even some security application where the device will self destruct if it detects it has been tampered with. Pretty awesome.

I’ve also made a note to look at Oracle Event Processing for Java Embedded – a server-side esque solution that contains Jersey, JavaDB for use in an embedded controller so it can evaluate business logic and decide what events should be forwarded on to a real server to notify the world.

My questions about tooling moving forward are answered positively. Netbeans and ME development should be moving off windows only in the future which will make a lot of MelbJVM members happy. The number of other mobile platforms and CPU’s supported is growing. The JVM can be adapted easily enough to run on other processors. (Edit: Just recently a MIPS platform build of Java 8 became available.)

In short JavaME – production ready, tested and certified.

The subtle lesson of the Java track for the day that keeps coming in questions and hallway chats is that keep your Java up to date to take advantage of the latest processor architectures. If you have some app server stuck on Java 6 or 7, those JVMs can only optimise for the CPUs that were in service at the time. If you buy new servers and deploy your old JVMs to them, even if the servers have new instructions sets, the older JVMs won’t be able to take advantage of them. This goes for SE, ME and EE. If you want speed, the best way to optimise is to bring your JVM stack up to date. All the JVM leaders I spoke to had stories of a significant chunk of their production software still on 7 or even 6. All were welcoming of any info that we could bring back to our communities to help our members with ammunition to convince corporate IT of the necessary upgrades. (Special mention to Sunny of the Hong Kong JUG / Goldmann Sachs for this insight)

Dinner

Oracle treated us to some delicious Cantonese at the Jumbo floating restaurant. There is a ferry to shuttle you from the pier to the barge and back again.

Day 3 – Back to the community

This was a single stream and was applicable to everyone:- What was the support offered by Oracle to application user groups and JUGs, and how could we take advantage of it.

More 4 minute pitches

Some more interesting 4 minute pitches arose. For me the most intersting were:

about how the Aussie Peoplesoft/JD Edwards group arranged their exhibitor hall for their upcoming conference based on lessons learnt from a professional showroom designer.
Marshall from the Chinese JUG, GreenTea, talking about how JUGs can collaborate. Their examples certainly inspired me to seek out their JUG and given MelbJVM has already used Google Hangouts to have remote presenters, having us presenting at their JUG and vice versa is a no brainer, especially now that we’ve made the connection in person.
Pearl from the Taiwanese Oracle UG community talking about the events they held to help build member interaction. Ukelele playing is popular over there so the ukelele will sit in my mind. The emphasis was on social events rather than techy ones and certainly influenced me opinion on this.

Support

Oracle can send out speakers to your user group event with enough notice, say about 4 months in advance to be able to lock the event in speakers calendars. They prefer to find speakers that live in the region rather than fly them out from HQ (USA) where possible. Oracle is a global company and there are a lot of experts in all corners of the globe. I know from experience there are a number of Oracle employees with significant contributions to Java libraries and tooling based in Australia already. That said this is not a hard and fast rule and if you do want the Java luminaries of this world to present in person at your meetup, you just need to make sure you can get enough attendees to make that worthwhile. The more cities you can get involved the better the chance of increasing that number. The recent Java 8 China tour was a great example, 400 people in 4 cities. The upcoming Australia tour beginning this week should be close – at least on a per capita basis 😉

Oracle can also hold events at their facilities with enough notice. There is some prep work to be done, 4-6 weeks notice again, and finalising the attendee list 48 hrs prior, the event can’t be co-branded with other organisations, but it is good to know it is available.

Finally Oracle can help promote the event on the User groups and community sections of the Java home pages. That way when people go to download an update to Java or are just browsing for news, the details of the event will come up in the side bar.

The people to get in touch with are the user group events team at Oracle.

OTN and Community

The next talk was all about the Oracle Technology Network, ACE champions and Java Champions.

Oracle have the ACE program for people in the applications side. These are people nominated by Oracle that contribute to the community – writing blogs, authoring books and speaking publicly in support of Oracle and their products.

For Java though, the situation is a little sweeter. There is the Java Champions Project that have the same level of respect from Oracle as they give to ACE’s. Like an ACE, a potential champion has to be a leader in the community and a tech luminary. They could also have used Java technology to aid a humanitarian effort or to evangelize others. An example of someone who was down in Melbourne recently, Charles Nutter, lead of the JRuby project, is a Java Champion for his contributions to the JVM platform (invoke dynamic and continuing work on FFI).

The neat thing that is different to the ACE program is that Oracle doesn’t have a say in who becomes a champion, the Java Champion community peer reviews and decides who to let in. Its an independent group and Oracle acknowledge that there are going to be differing opinions. Unlike ACE’s, Java champions can write anything about Oracle be it positive, neutral or negative to Oracle’s position on any issue. I for one applaud Oracle for that.

We need you

An interesting statistic was that the AsiaPacific region has the largest number of registered OTN users outside of the USA. However the number of Java Champions in the region is quite small, only 7% of the global champion community. We need more Java Champions here, because without them, its more difficult to nominate more from the region to add. It was a subtle hint to all JUG leaders that we need to develop champions in our region.

Keeping in touch

Finally there is some more things Oracle will be doing in the coming months to make it easier for user group leaders to keep in touch. In the meantime you can check out the twitter hashtag @oracleugs, and look at the user group site IOUC.org .

If you are a JUG lead and haven’t gotten on to it already, sign up to the Jug-leaders list (register for a java.net account 1st), create a placemark for the global map so your JUG is searchable on the Java.net site and Oracle have visibility of you, and take a look at the getting started pages on the wiki

Big Takeaways

TL;DR version of this experience

I expected to meet a lot of JUG leaders in the region. They are actually nice guys and girls and I perceived them as like minded fellows – which is a nice bonus.
There is a world outside of Java in Oracle and we’ve got as much to learn from the applications user community as they do from ours.
Asia Pacific has a huge base of Java developers. There is untapped potential with lots and lots of talent that should get out to the rest of the world.
There is a language barrier, but its small most of the time. Meeting people in person helps alleviate these barriers.
Oracle are a huge corporate entity with a big user group supporting heart. If you are still living in the past days of the corporate takeover from Sun, that process is now truly done and dusted. Oracle can and does put more effort and dollars into community than Sun ever did.
Java Champions get a lot of flexibility afforded to them by Oracle.
APAC needs Java Champions. Do you know any?
Oracle can help out with promotion, speakers and other things for your events so dont be a stranger and get in touch early when you are planning an event that needs their help.
If user groups can partner with each other to drive a tour or conference Oracle will come to the party.
If you aren’t on IOUC.org, don’t have an OTN account, not on Java.net or the JUG leaders list, then Oracle can’t include you in events like this. So make sure you get on it!

I’m assisting with the planning for the next International Oracle User Community (IOUC) Leaders conference in September, just before OpenWorld / JavaOne in San Francisco, so to all JUG leads in the APAC region, any feedback on this event or things you want raised in future events, please let me know and I’ll do my best to make sure you’re heard.

Also, the next AsiaPac OUC will is tentatively scheduled for April next year. I’ll post out details via the usual social networks as details are firmed up for that.

April 28, 2014

Git proxy woes? Get a corkscrew

A quick note that I’ve got a very particular proxy at work. Its also an authenticated proxy which gets in the way of a lot of programs that need the Internet.

git allows you to specify a http / https proxy config but not all authenticated proxies work with these.

This blog ‘Accessing git with an authenticated proxy’ explains how to get around that by using a proxy command to launch corkscrew. Its actually quite successful, I already used corkscrew for ssh auth so this wasnt too much config at all.

Just remember to take out your git config for your failed attempts at http.proxy and https.proxy

git config --global --unset http.proxy
git config --global --unset https.proxy

January 1, 2014

Weeks turn into months, months turn into years…

It’s been almost two years since my last blog. A lot has happened in that time so I wanted to do a status update – No I’m not dead yet!.

The thing that has taken the most time has been starting the Melbourne Java and JVM users group, affectionately known as MelbJVM.

The group has grown from 14 people in our first meeting in March 2012 to over 500 members today with regular sponsors, venues. We’ve had both international and local speakers, partnered events with other tech communities, and hosted workshops. There is now an organising committee to help drive things as well. I have to say doing something for the community through MelbJVM has been the most rewarding thing I’ve ever done. I’d encourage anyone reading this with an inkling for wanting to help any community, do it, you won’t look back!

2013 was a great year personally as well

This happened

That happened

and then happened again the next day

These trips happened

as well as a massive conference trip in September

These gigs happened:

Puscifer
A Perfect Circle / Kyuss / Slayer at Soundwave
Tool on my birthday weekend (refer to photo above)
Black Sabbath in Melbourne
Deftones / Tool / Black Sabbath at Ozzfest Japan
Karnivool

I also had the pleasure of being Best Man at this event

Other than that, long time friends have had babies or gotten married or engaged and we couldn’t be happier for them.

Here’s to a prosperous 2014 everyone!
(2013 will go down for me as a great year and one that sets the bar high)

March 14, 2012

Get your CTRL+ALT+Fx keyboard shortcuts back in Ubuntu 11.10

I’m now using Ubuntu on my work desktop (nice change). My favourite IDE, IntelliJ Idea has a very useful shortcut CTRL+ALT+F7 to Show Usages of anything under the cursor. Its one of the most simple yet powerful features of the app. The unfortunate problem is that X11 uses CTRL+ALT+F<1..6> to switch between consoles and CTRL+ALT+F<7..9> to switch between desktops and it wins, so IntelliJ doesn’t respond to that useful shortcut anymore.

Tucked away in the Keyboard Layout control panel is an Options… button which brings up an options dialog with various options to configure your keyboard strokes – different to the normal Keyboard bindings you find in the normal Keyboard settings. These are more how you want the keys to be mapped rather than mapping them to functions. Under Miscellaneous compatibility options you’ll find ‘Special keys (Ctrl+Alt+<key>) handled in a server‘ and ticking this gives your shortcuts back to the desktop apps.

Most *nix software wouldn’t assign shortcuts to the ctrl+alt+f<n> because its an accepted convention, but for software that works on multiple OS’, its nice to take this back.