adam bien's blog

10.000 Thin WAR deployment cycles or IoT with pure Java EE at IBM 📎

Tobias, please introduce yourself

My name is Tobias N. Sasse, from Cologne (Germany) and I am a Software Architect and Managing Consultant with IBM Services. I support our enterprise clients, mainly from the insurance industry, to succeed with their digital transformation journeys. I have a focus on developing industry platform solutions, which enable our clients to adopt new capabilities as a service and accelerate a fast go-to-market with innovative technology-driven products. This sounds quite abstract, but I will give an example of what my team does later when we talk about my projects.

In my daily job, I am leading a team of incredibly talented software developers, quality engineers, and business consultants. My focus in the group is to provide the general direction for the products we are working on and design an architecture blueprint, which ensures that we meet the functional and non-functional requirements. Whenever possible, I enjoy writing source code and evaluate new technologies.

My technical focus lies on distributed and scalable systems. You can find me on Linked.in, and my twitter handle is @tnsasse.

What are you building with Java EE?

IBM has an initiative around aging at home with the aim of keeping seniors safe in their own homes longer. I am leading the effort of building such a platform. We offer this industry platform, to our clients as a service, and it combines Internet of Things technology, with a smart analytics engine in the cloud, and professional care services from our partners from the insurance- and healthcare sector.

Imagine a family, where granny Martha lives alone a couple of hundred miles away from her son Robert and his kids. Robert is concerned about the wellbeing of his mother, with all the risks that evolve around growing older. At the same time, he is not able to visit as often as he'd like, to check up on her. Martha enjoys her home and couldn't imagine moving to a nursing home or assisted-living facility.

Now with our product, Martha and Robert could get help from their insurance provider. Professional technicians would visit Martha's home and install a couple of minimal-intrusive sensors (from one of our partners) in her home, which monitor her daily activities. For Robert, we have a mobile app that lets him check the status of Martha's apartment and informs him whenever the system detects a dangerous situation. If something really bad happens, the platform will detect this through the analytics and will notify Robert as well as a connected care provider. The care provider will send a care specialist to Martha's home to assist. They can also send an ambulance, firefighters or the police. With a little box in her home, Martha can even talk to the call center of the care provider whenever our analytics detect an emergency.

The role of IBM in all of this is to establish and nurture the business ecosystem of partnering companies and to provide the technology and algorithms behind the platform. To do so, we offer a set of APIs, that enable our partners to integrate their services or processes with the platform, like to alert a call-center about an incident, or to dispatch a maintenance order to the sensor technician.

IBM is also bringing its cognitive analytics capabilities to the table, that help to make sense of the raw stream of sensor data and detecting hazardous situations, for each home. The IBM Cloud offers the necessary resources to handle this enormous flow of data and to operate all of the services in a real-time, secure and high-available manner.

Except for the mobile apps and a management web interface, we implemented the platform in pure JavaEE 7. We also applied a micro-service architecture to our system, which helps us to individually scale or replace parts of the system very well with minimal impact on our critical uptime.

Can you share with us some geeky numbers like, e.g. TX per seconds, heap sizes, thinnest WARs, etc -- whatever Java EE devs might find interesting.

We currently have ~10 micro-services. Our WAR sizes are a little blown up because we have to rely on a bunch of connector libraries for proprietary systems. Overall though, we are pursuing a thin-WAR approach, and none of our apps are heavier than 5 MB.

Our app servers comfortably run in a 500 MB heap configuration. We default to 1-2 GB though and prefer to scale horizontally if we need additional capacity. We are currently in our 26th (SCRUM) sprint, and overall we have deployed ~ 10,000 times on our development stage, not counting any local deployments. Deploying multiple times a day, the thin-WAR approach and fast app server boot/deployment times pay off for us. I am happy that we don't need to upload 60 MB of framework libraries to the cloud all the time.

Our design is built for scale, as the platform is supposed to handle not only Germany but most of Europe, so we are planning on a 7 figure apartment count, which is our primary measurement KPI. Each instance of our micro-services can sustain a couple of hundred transactions per second. Again this is not easy to measure, as our transactions range from trivially simple to quite complex.

Are you happy with Java EE so far? Is Java EE productive?

IBM has a rich history with Java EE, and most people will know, that we are actively contributing in the expert councils and as an application server vendor. We even have our own Java SDK (IBM Java Technology Edition) which is optimized and production-proven for a variety of architectures. Our JDK was recently open-sourced as OpenJ9, by the way.

From that perspective, Java EE is very much at home here. My team is incredibly productive with Java, and it provides the robustness we seek for in an application like ours. It's not only a rational technology choice, but also a lot of fun to work with the EE7 APIs, and all the goodies that came with Java 8. We still have to look into EE8 and Java 9 for that matter and are also looking into the Micro-Profile APIs, which we would love to integrate soon.

Which application servers, tools or IDEs are you using?

Of course, we are relying on IBM WebSphere Application Server Liberty Profile, which combines the best of both worlds: a small footprint, fast boot- and deployment times, enterprise-grade reliability and features. Additionally, our operations team can rely on 24/7 support from our global support colleagues, in case anything happens in production. WebSphere Liberty was also open-sourced recently as OpenLiberty and is free to use for everyone, and I have seen that you predicted it to be the killer app server for 2018.

We are deploying our apps on the IBM Cloud, which is an entirely automated process triggered by our DevOps pipeline. We host our source code in an IBM internal GitHub Enterprise repository, build it with Maven and deploy it via the IBM Cloud APIs. After the deployment, we automatically execute regression and system tests to ensure that we meet our quality standards.

My development team is using a well-known issue system for our agile sprint planning, and our IDE is based on Eclipse, which given the long history of IBM with the Eclipse IDE might not come as a surprise, too.

Our system has to handle a significant amount of IoT data, and we rely on the Watson IoT Platform to manage all the MQTT connections, security and data streaming. Of course, we have to make sense of the data we receive, and our data scientists are working with IBM DataScience Experience and Watson APIs to develop their machine-learning and cognitive analytics models.

Database-wise, I am a big fan of Db2 since my time at the IBM Almaden Research Lab back during my studies, where I had a chance to look under the hood and see some of the fantastic technologies it employs to optimize queries and workloads. So, we are using IBM Db2 on Cloud for most of our data, as it performs remarkably well and offers a high amount of security at the same time. We also employ IBM Cloudant for NoSQL document-based JSON data, wherever we need a high amount of throughput without transactional security.

You are using the Boundary Control Entity pattern to structure your applications. What were your experiences, challenges, and findings so far?

Yes, we do use the BCE pattern, although my team prefers a different package naming structure. We use ".api" for the boundary layer, ".service" for the control layer and ".model" for the entities. For me, the BCE pattern is not about the naming - it's more a guideline to stay within best practices: to keep the source code aligned with the business problem it tries to solve and to eradicate the use of those ".commons" or ".util" packages that the industry grew so fond of. I think that the pattern helps us to write more readable and streamlined code. The lean code helps us to produce better quality, too many bugs hide in unnecessary abstractions, and wrongly applied design-patterns.

How important are standards for you? Does your application depend on application server specific APIs?

IBM is a big supporter of open standards, and this is not only shown by our contribution to Java but even more in our cloud. If you take a closer look at the IBM Cloud you will see, that we base on OpenStack, Cloud Foundry, Kubernetes, and Linux Containers. For our specific application, standards are essential for our public APIs, so that we are on the same page with our partners in the industries. We embrace the REST paradigm, JSON, Swagger, OAuth 2 and OpenID Connect for example. As we are not planning on moving to a different application server, or database, we make full use of the features the platform offers. In practice, the majority of what we do is plain Java EE standard though.

As a team leader, a significant benefit of standards for me is, that I can quite efficiently communicate skills with HR. If I ask for a Java EE specialist, they know whom to look for, and I am quite sure that the person they identify will have the skills I need.

Which Java EE APIs are you using in your products?

We do use JAX-RS and JAXB for our boundary layer (HTTP/REST with JSON). Our control layer is implemented with EJBs plus some CDI, and we do (most of) our entity layer with JPA. Other than that we do use filters from the Servlet spec, for authentication, CORS, and alike. We have a couple of interceptors as well.

Did you have the chance to chat with other attendees during the airhacks? If yes, what was the most interesting project / problem / situation in the workshops?

Oh yes, I had a lot of interesting chats with attendees all over the world. One story stuck with me: we were talking about how crazy complex we developed software in the past when standards like we know today weren't around that much. One of the attendees told me, how he implemented his own database, with built-in versioning and home-grown encryption capabilities in Java. It sounded like a terrible idea in retrospect, and we had a good laugh together, but I guess these are the projects you can learn a lot from after all.

Can you share any resources with us?

I pointed out that some my favorite IBM products just went open-source. You can find OpenLiberty at openliberty.io which is my favorite EE app server, and the IBM JDK seems to be less known, but is a fantastic piece of engineering and is now released as Eclipse OpenJ9

Concerning the Java ecosystem, a project I grew quite fond of is the MicroProfile initiative, which released a bunch of handy APIs, that will hopefully find their way into a future EE version. The beautiful thing about OpenLiberty is that you can just add MicroProfile, with a single line in your server configuration file.

Of course, I'd encourage everyone to take a look at the IBM Cloud, which is based on open standards, so no vendor lock-in, provides Cloud Foundry, Containers, Kubernetes, Watson APIs and a lot of useful services. We also offer a free tier for many services, so you can get started without a credit card.

For your German-speaking readers, I'd also like to point towards the IBM THINK Blog DACH covering interesting articles on how technology changes companies and society in Germany, Austria, and Switzerland.

Tobias, thank you for the interview!