Monday, 17 December 2012

Infinispan 5.2.0.Beta6 is out!

5.2.0.Beta6 brings a new batch of fixes around Non-Blocking State Transfer, Map/Reduce and command line interface.
But it's not only that, it also brings a bran new pice of functionality: support of concurrent updates for non-transactional caches(ISPN-2552) . Prior Infinispan 5.2.0.Beta6, there was a high chance for a deadlock to occur when two threads concurrently update the same key. This caused significant performance costs and throughput degradation, linear to the amount of contention. This functionality is enabled by default even though a compatibility mode is still available. You can read more about it here.

For a detailed list of all the issues fixed please refer the release notes.

You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Monday, 10 December 2012

Back from JUDCon China 2012

JUDCon China was held for the first time last week in Beijing, and it was a blast!

I had never been to China before, let alone a conference there, so it was interesting to compare it with other conferences around the world. The use of simulaneous translators meant those giving presentations in English had to pace themselves a bit more so that translators would have time to catch up. There's was a good bunch of presentation given by native Chinese speakers too, so the audience got the chance to attend more interactive sessions too.

During this two-day developer conference, I was showing Infinispan's capabilities as a powerful cache bulding a transactional EE application that neeeded to scale up live. In this presentation, which combines Infinispan with JSF and CDI, I showed how to go from a basic temporary cached, to a clustered cache that used consistent hashing to distribute data, showing the ability to scale up and failover. This presentation which was given shortly after the keynote on the first day generated a lot of interest in the audience, with a lot of users wanting to find out how we compared with other existing cache and data grid providers. This was a great opportunitiy to introduce the rest of Inifnispan talks that were happening that day and the day after, where they could learn more Infinsipan's other features as a data grid: i.e. querying, geographic failover...etc.

My second presentation that same day was about querying Infinispan based data grids. The room was packed for this one, and in the presentation I talked about how Infinispan's map/reduce functions can take advantage of the paralellism available in cluster to resolve basic queries, and how Infinispan's contents can be indexed using the query module and queried via Hibernate Search or Apache Lucene APIs. Finally, I did a little overview of higher level APIs offering further querying possibilities (i.e. ModeShape, Hibernate OGM). As pointed out by Ales, the lead of CapeDwarf team, I somehow forgot to add his project, which offers the possibility of running Google App Engine on top of JBoss Application Server, which uses Infinispan and offers different querying possibilities too. Don't worry Ales, we'll sort it out for this presentation's next outing :)

For those who attended, the presentations should be uploaded to the JUDCon China website within the next few weeks.

Not all was presentations though, we also had the chance to socialise with attendees and other Red Hat employees in China. After all talks finished the first day, Jim Ma and Yong Yang did a fantastics demo of a cluster of JBoss Application Server 7 instances running on Rasperry PIs, and the audience had the opportunity to win one a Rasperry Pi too!

If you're thinking of going to China, you can't leave without trying a hot pot place! On Friday night we went to a Hai Dai Lao Hot Pot restaurant for food, and from a culinary perspective, it was the best food I had during my China stay. Hot Pot restaurants have this concept of cooking different meats, vegetables and noodles in two hot pots, one spicy and the other not. On top of that, you get the chance to mix up some cold sauces yourself (sesame oil, coriander, nuts, chillies,,,etc) and mix that with the cooked meat/veg. Unfortunately, one of our colleagues who had a nut allergy had to be rushed to hospital, but it was all Ok in the end :).

From here I'd like to thank Cindy Dong, Jervis Liu, Jim Ma, Christina Lin and many others that helped us, Aliens (according to Chinese Immigration prospects), feel like at home :). Can't wait to get back to China.


Monday, 3 December 2012

Infinispan 5.2.0.Beta5

Hi Infinispan users,

5.2.0.Beta5 contains the usual batch of fixes especially around Non-Blocking state transfer functionality, Map Reduce and the CLI.

Functionality-wise, it is now possible to read/write data concurrently to the same remote cache both via a RemoteCacheManager and a RemoteCacheStore by enabling "rawValues" in the latter. This is the final item that was needed to enable us to implement Rolling Upgrades for remote caches. Because of this feature the HotRod protocol required a couple of extensions and therefore its version has been bumped to 1.2

For a detailed list of all the issues fixed please refer the release notes.

You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Wednesday, 21 November 2012

Back from Devoxx

This year - as always - Devoxx was a great conference! Not only the quality of the presentation (including quite some delivered by my fellows from Red Hat)  but also the chance to meet and discuss with industry experts. And the Hackergarten was one of the this great networking places.
We had a long chat with Marius Bogoevici on a nice extension of the Ticket Monster Tutorial for Infinispan, so that users looking for an sample integration application would have a good starting point. Also Duncan Doyle  contributed a nice demo of the cross-site replication functionality added in Infinispan 5.2 and Guillaume Scheibel implemented a Mongo DB cache store extension: awesome stuff!
All in all great conference and very good chance for us to get in touch with our community!

Monday, 19 November 2012

Infinispan 5.2.0.Beta4 is out!

5.2.0.Beta4 contains a handful of fixes around mainly around Map/Reduce and Non-Blocking state transfer functionality. For a detailed list of all the issues fixed please refer the release notes.

You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Infinispan @ Devoxx 2012 Hackergarten

Just came back from Devoxx, and once again it didn't let me down! Great conference, with awesome talks, and fantastic networking opportunities (hackergarten, party...etc). As far as Infinispan is concerned, I joined the Hackergarten on Tuesday morning to try lure some contributors into our project :).

One of the most promising opportunities came from Alex Soto, who's founder and lead of NoSQLUnit, which is a JUnit extension that helps you write NoSQL unit tests. He already has support for a number of NoSQL engines and he was explaining me the challenges of supporting multiple engines. We also discussed the possibility of supporting Infinispan as well :D.

During the Hackergarten I also met Andrés Almiray, which is the Griffon founder, and we briefly discussed the possibility of adding integrating Infinispan's Hot Rod client with Griffon clients. Unfortunately we didn't have time to get into some coding, but since he lives close-by and he organises Hackergartens in Basel, I might pop in next time around and sit down with him to work on this integration :).

Can't wait for next Devoxx!!


Tuesday, 13 November 2012

Devoxx here we come!

If you're interested in data grids and especially in general and measuring data grids performance in particular then come to my presentation at this year's Devoxx.
Manik Surtani is getting funny (literally!) with a gig suggestively named "Will you Map/Reduce my cloud" and Galder Zamarreno brings the polyglot JBoss at Devoxx.
We'll also be around the Red Hat boot in case you want to have an Infinispan related discussion or just to say hi.


Monday, 5 November 2012

Infinispan coming to JUDCon China 2012

The Infinispan team will be well represented in the forthcoming JUDCon China 2012 conference to be celebrated in Beijing on the 29th and 30th of November with no less than 9 talks!

We'll be covering all sorts of topics:
- Building transactional Infinispan applications
- Performing rolling upgrades for Infinispan based cluster
- Scaling Hibernate/JPA applications using Infinispan-based second level cache
- Effective querying of Infinispan data grids
- Measuring performance of data grids
- ...etc

So, if you're interested in data grids, caching or Infinispan, come and join us!! It's gonna be a fun a couple of days. Can't wait :)


Tuesday, 30 October 2012

Infinispan 5.2.0.Beta3 is out!

We're one step closer to the final.
This release contains several critical bug fixes around transaction consistency during state transfer, some performance enhancements and various other bug fixes. For a detailed view of what has been fixed please refer to JIRA.

You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Sunday, 14 October 2012

Infinispan 5.2.0.Beta2 is out!

Infinispan 5.2.0.Beta2 contains a handful of bugfixes especially around the new Non-Blocking State Transfer functionality. For a detailed view of what has been fixed please refer to JIRA.

You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Tuesday, 9 October 2012

Infinispan @JBoss One Day Talk in Munich!

If you happen to be in Munich for this year's JBoss One Day Talk and Infinispan is something you want to know more about then come and see the Infinispan presentation! I'll also be hanging around so just ping me and come for a chat.


Wednesday, 3 October 2012

5.2.0.Beta1 released!

This is the complete list of bugs/features released. Highlights:
  • Various enhancements for the distributed executor framework (ISPN-2287ISPN-2286ISPN-1513)
  • A new, faster, more efficient and elegant implementation of the AsyncCacheStore - big thanks to Karsten Blees for contributing it!
  • The default transaction enlistment model has changed from "xa" to "synchronization". You should only care about this if you're using recovery (ISPN-1284).
  • The ability to mark a site as offline after a certain number of request (ISPN-2319)
You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Wednesday, 19 September 2012

5.2.0.Alpha4 brings cross-site replication into Infinispan!

Besides other enhancements and fixes, this release brings the first implementation of the cross-site replication functionality in Infinispan. In other words, you can now use Infinispan for backing up your data across geographically distributed sites or migrate your data where your users are (follow the sun).
More about the x-site replication functionality here.
You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Wednesday, 5 September 2012

Infinispan Arquillian Container 1.0.0.CR1 released

Infinispan Arquillian Container is an extension to Arquillian that provides several ways to interact with Infinispan, either with a standalone Infinispan server or just with Infinispan libraries. This extension can also communicate with JBoss Data Grid server via JMX.

It was released as Maven artifacts in JBoss Maven Repository. It is located at . More information on how to set up and use the repo can be found at

What does this Arquillian extension offer to you? Let me describe all aspects of this extension one by one.

Developing tests with standalone Infinispan server

When testing, you might want to automatically start the Infinispan server before the test and stop it afterwards. This can be achieved by configuring infinispan-arquillian-container via Arquillian's configuration file. The following is a subset of attributes that can be specified and thus passed to the Infinispan server during startup: masterThreads, workerThreads, cacheConfig, jmxPort, ... The complete list can be found in (all private fields).

NOTE: Examples are not a part of the release, only libraries are. In order to check out examples provided with the project, one has to clone project's repository: Examples are located in the respective sub-directory.

The configuration file then looks similar to the following:

Whether these two Infinispan servers are clustered or not depends on the configuration passed to them via cacheConfig (file path) attribute or their default configuration (when no config. file is passed). The configuration in arquillian.xml file just says: "Start these two instances with whatever configuration is passed to them".

Complete example:

When we tell Arquillian to work with Infinispan server, we can inject RemoteInfinispanServer object into our test. Such an object provides various information about the running Infinispan server. For example, we can retrieve a hostname and HotRod port and use these pieces of information to create a RemoteCacheManager instance. Besides that users are allowed to retrieve information available via JMX from the server like cluster size, number of entries in the cache, number of cache hits and many more.

Complete example:

Vital dependencies required for the test to run are:


Not only with standalone Infinispan server can Infinispan Arquillian extension work.

Developing tests with JBoss Data Grid (JDG) server

This time, the properties in Arquillian's configuration file are different and correspond to properties of JBoss Application Server 7. The most important property is again the path to the server (jbossHome).

Are you interested in what the test looks like? It looks completely the same as tests for standalone Infinispan server, you just have a few more attributes available. JDG server usually starts all three endpoints (HotRod, Memcached, REST) at the same time while for the Infinispan server you have to specify which end point should be started. Furthermore, Infinispan server does not have the REST endpoint available out-of-the-box.

As a result, you can call the following methods with JDG in one single test.


The difference is, of course in dependencies. Instead of a handler for standalone Infinispan server, one has to use a handler for JBoss AS 7. The dependencies then look like this:

Testing Infinispan libraries

Sometimes we don't want to use a standalone server. Sometimes we want to test just Infinispan in its basic form - Java libraries. Infinispan has been under development for years and during that time, lots of tests were developed. With tests come utility methods. Infinispan Arquillian Container enables you to leverage these utility methods and call them via an instance of DatagridManager. This instance can be easily injected into a test, no matter which test framework (TestNG, JUnit) you use.

DatagridManager class can be found at

Can you see the advantage? No? Let me point out some useful methods available in the manager.

List<Cache<K, V>> createClusteredCaches(int numMembersInCluster, String cacheName, ConfigurationBuilder builder)

- creates a cluster of caches with certain name and pre-defined configuration

void waitForClusterToForm()

- helps to wait until the cluster is up and running

Cache<A, B> cache(int index)

- retrieves a cache from certain node according to the index

Cache<A, B> cache(int managerIndex, String cacheName)

- retrieves a cache with that name

void killMember(int cacheIndex)

- kills a cache with cacheIndex index

AdvancedCache advancedCache(int i)

- retrieves an advanced cache from node i

Trancation tx(int i)

- retrieves a transaction from node i

TransactionManager tm(int i)

- retrieves a transaction manager from node i

...and much more.

The following test can be found among other examples in the GIT repository.

Required dependencies:

org.infinispan:infinispan-core:jar:5.1.5.FINAL:test  -  users should replace this version with the one they want to test

Infinispan Arquillian Container was tested with Infinispan 5.1.5.FINAL and JDG 6.0.0.GA. Nevertheless, it should work smoothly also with other not-very-distinct versions. I'll be updating the project to work with newer versions of both Infinispan and JBoss Data Grid.

Tuesday, 4 September 2012

Speeding up Cache calls with IGNORE_RETURN_VALUES invocation flag

Starting with Infinispan 5.2.0.Alpha3, a new Infinispan invocation flag has been added called IGNORE_RETURN_VALUES.

This flag signals that the client that calls an Infinispan Cache operation () which has some kind of return, i.e. java.util.Map#put(Object, Object) (remember that Infinispan's Cache interface extends java.util.Map), the return value (which in the case of java.util.Map#put(Object, Object) represents the previous value) will be ignored by the client application. A typical client application that ignores the return value would use code like this:

In this example, both cache put call are ignoring the return of the put call, which returns the previous value. In other words, when we cache the last login date, we don't care what the previous value was, so this is a great opportunity for the client code to be re-written in this way:

Or even better:

Thanks to such hints, Infinispan caches can behave in a more efficient way and can potentially do operations faster, because work associated with the production of the return value will be skipped. Such work can on occasions involve network calls, or access to persistent cache stores, so by avoiding this work, the cache calls are effectively faster.

In previous Infinispan versions, a similar effect could be achieved with flags with a narrower target and which are considered too brittle for end user consumption such as SKIP_REMOTE_LOOKUP or SKIP_CACHE_LOAD. So, if you're using either of these flags in your Infinispan client codebase, we highly recommend that from Infinispan 5.2.0.Alpha3 you start using IGNORE_RETURN_VALUES instead.

Cheers, Galder

Saturday, 1 September 2012

Infinispan 5.2.0.Alpha3 is out!

There are releases and releases. And this one is a big one, containing a bran new state transfer functionality. Designed and implemented by Dan Berindei and Adrian Nistor, the new Non Blocking State Transfer (NBST) has the following goals:
  • Minimize the interval(s) where the entire cluster can't respond to requests because of a state transfer in progress.
  • Minimize the interval(s) where an existing member stops responding to requests because of a state transfer in progress.
  • Allow the performance of the cluster to drop during state transfer, but it should not throw any exception
Curious to see the magic behind it?  This document is here to explain you NBST's internal.

Besides NBST this release brings some other goodies:
  • A new IGNORE_RETURN_VALUES flag to help reduce the number of RPC calls and increasing performance (to be discussed at large by Galder Zamarreño in a following blog post)  
  • A revamped and much nicer configuration for submodules such as cache loaders. More about it in  Tristan Tarrant's blog
  • for a complete list of the fixes/enhancements refer to JIRA
Another new thing this release brings is a change in versioning: we've aligned to JBoss' release versioning pattern. So the name is now Alpha3 vs ALPHA3(as per the old naming pattern). More about the reason for doing that in this blog post.
The complete list of issues/improvements addressed in this release is available in JIRA. As always, please give it a try and let us know what you think on the forums, irc or mailing lists!


Friday, 31 August 2012

Configuration overhaul

Infinispan 5.2 will sport a much needed configuration overhaul which will affect both the programmatic builder API and the declarative XML parsing.

As you all know by now, 5.1 introduced a new fluent builder-based API with immutable POJOs for configuring Infinispan's core. This coolness however was not extended to all the extra modules available for Infinispan (and there are quite a few of those), leaving them with a simple untyped key/value properties-based configuration. This was especially visible (and painful) when configuring the cache loaders, some of which have a plethora of parameters and options.

In 5.2 modules become first-class citizens and can provide their own builders and can take care of parsing their own XML for which they can provide a custom schema (for editors/IDE which provide content-assist). Modules can retrieve information from either the GlobalConfiguration or the per-cache Configuration objects via the T modules(Class<T> moduleClass) method.

Loaders and Stores also get this treatment. Look at the two before and after configurations below for configuring the JDBC Cache Store.



You will be able to check-out these features in Infinispan 5.2.0.Alpha3. Bear in mind that at the time of writing not all cache loaders have been migrated to this new configuration style, but they should all be complete by the time 5.2.0.Final is released.

If you want to learn how to extend Infinispan's configuration for your own modules, head over to ExtendingInfinispansConfiguration which should provide all the information you need

Thursday, 30 August 2012

HotRod C# client Beta1 is out!

Thanks to a sustained effort from  Sunimal Rathnayake, the C# HotRod client has evolved quite a bit:

  • the public API was polished and finalized
  • the client was upgraded to 2nd level of intelligence: that means that it can automatically piggyback cluster's topology information from the servers. E.g. if a new server is added, the client is made aware of it and can start balancing requests towards that server
  • a pluggable load balancing policy was added (defaults to round robin) 
  • various other bug fixes backed by an growing test suite
We've also set up a development setupiv document to speed you up in case you want to take a look at the code or contribute. 
You can download it from here - please give it a try and don't hesitate to post your comments to our forums, the mailing list  or contact us directly on IRC for a chat!


Wednesday, 22 August 2012

Infinispan project versioning change

Mainly for consistency reason, starting with the next Infinispan releases, we'll switch to JBoss' release naming conventions. In practical terms this means that the names of the releases(Maven artifacts, JIRA) would be different.

Old release name New release name
5.2.0.ALPHA2 5.2.0.Alpha3
5.1.0.BETA1 5.2.0.Beta1
5.1.0.CR1 5.2.0.CR1
5.1.0.FINAL 5.2.0.Final

The names of the releases in JIRA have also been changed startting with 5.1.2.Alpha3.


Tuesday, 21 August 2012

JavaOne 2012, here I come

Almost time for JavaOne 2012, the biggest geekfest this side of Betelgeuse Five.  I will be speaking again this year, and this time I have both a conference session as well as a birds-of-a-feather session.  If you're going to be at JavaOne, have an interest in in-memory data grids, distributed caching, performance, scalability and NoSQL, do drop in to one of my talks.

The first one is the BoF - early evening on Monday, the 1st of October, where I host the BoF titled "JSR 347, Data Grids, and NoSQL". This BoF is an interactive session targeted at anyone with an interest in standards around data grids and NoSQL, where I would like to discuss the overlaps, convergence and divergence, and of course standards around such technologies.

The second one, a conference session titled "Making Apps Scale with CDI and Data Grids"is a demo-driven talk on building applications to make use of data grids and distributed caching for performance and scalability, using the popular CDI programming model.  Some familiarity with CDI is expected.

In addition to the above two talks, I will also be speaking at the Red Hat booth on data grids, NoSQL and related subjects.  So do drop by and say hello!


Monday, 20 August 2012

Infinispan 5.1.6.FINAL is out!

We've just completed a maintenance release for Infinispan 5.1.x branch. Besides other fixes, this release contains a critical fix for Hibernate integration, i.e. using Infinispan as a 2nd level cache in Hibernate ( a big thanks to Sanne Grinovero for looking into this).
You can download the distribution or the maven artifact. If you have any questions please check our forums, our mailing lists or ping us directly on IRC!


Wednesday, 15 August 2012

Infinispan - Data Grid Platform

For the past few months, I have been working with prominent technology blogger and author Francesco Marchioni on my first book ever.  Unsurprisingly, this is a book on Infinispan, written for Packt Publishing.

This is an entry-level book, covering high level concepts of distributed caching and data grids, before diving into practical, hands-on details of setting up, configuring and using Infinispan.  Monitoring a running grid via JMX and RHQ are also covered, as is a chapter on using Infinispan from CDI, the increasingly popular programming model for enterprise Java.

Grab the book - in print or electronic format - from Packt's website.


Tuesday, 31 July 2012

C# client for Infinispan - alpha release

A while ago I was announcing that Sunimal Rathnayake would start the work for a C# Hot Rod  client for Infinispan as part of the Google Summer of Code. After 2 months of heavy work Sunimal delivered an intelligence-one (basic client, interested in neither cluster nor hash information) implementation.
The release distribution can be downloaded from here. Besides the required binaries and doclets, is also contains a sample application showing how the client can be configured and illustrating the basic operations with the server. This and more are being described in the readme.txt file in the distribution root.

And there's much more on the way! Sunimal is working on enhancing the client to the intelligence-two level: topology-aware client, interested in cluster information - stay tuned!

Please give it a try and don't hesitate to post your comments to our forums, the mailing list  or contact us directly on IRC for a chat!


Wednesday, 25 July 2012

Map/Reduce improvements in Infinispan 5.2.0ALPHA2

As our MapReduce implementation grew out of the proof of concept phase (and especially after our users had already production tested it), we needed to remove the most prominent impediment to an industrial grade MapReduce solution that we strive for: distributing reduce phase execution.

Reduce phase prior to the Infinispan 5.2 release was done on a single Infinispan master task node. Therefore, the size of map reduce problems we could support (data size wise) was effectively shrunk to a working memory of a single Infinispan node. Starting with the Infinispan 5.2 release, we have removed this limitation, and reduce phase execution is distributed across the cluster as well. Of course, users still have an option to use MapReduceTask the old way, and we even recommend that particular approach for smaller sized input tasks. We have achieved distribution of reduce phase by relying on Infinispan's consistent hashing and DeltaAware cache insertion. Here is how we distributed reduce phase execution:

Map phase

MapReduceTask, as it currently does, will hash task input keys and group them by execution node N they are hashed to*. After key node mapping, MapReduceTask sends map function and input keys to each node N. Map function is invoked using given keys and locally loaded corresponding values.

Map and Combine phase

Results are collected with an Infinispan supplied Collector, and combine phase is initiated. A Combiner, if specified, takes KOut keys and immediately invokes reduce phase on keys. The result of mapping phase executed on each node is KOut/VOut map. There will be one resulting map per execution node N per launched MapReduceTask.

Intermediate KOut/VOut migration phase

In order to proceed with reduce phase, all intermediate keys and values need to be grouped by intermediate KOut keys. More specifically, as map phases around the cluster can produce identical intermediate keys, all those identical intermediate keys and their values need to be grouped before reduce is executed on any particular intermediate key.

Therefore at the end of combine phase, instead of returning map with intermediate keys and values to the master task node, we instead hash each intermediate key KOut and migrate it with its VOut values to Infinispan node where keys KOut are hashed to. We achieve this using a temporary DIST cache and underlying consistent hashing mechanism. Using DeltaAware cache insertion we effectively collect all VOut values under each KOut for all executed map functions across the cluster.
Intermediate KOut/VOut grouping phase

At this point, map and combine phase have finished its execution; list of KOut keys is returned to a master node and its initiating MapReduceTask. We do not return VOut values as we do not need them at master task node. MapReduceTask is ready to start with reduce phase.

Reduce phase

Reduce phase is easy to accomplish now as Infinispan's consistent hashing already finished all the hard lifting for us. To complete reduce phase, MapReduceTask groups KOut keys by execution node N they are hashed to. For each node N and its grouped input KOut keys, MapReduceTask sends a reduce command to a node N where KOut keys are hashed. Once reduce command arrives on target execution node, it looks up temporary cache belonging to MapReduce task - and for each KOut key, grabs a list of VOut values, wraps it with an Iterator and invokes reduce on it.

Reduce phase

A result of each reduce is a map where each key is KOut and value is VOut. Each Infinispan execution node N returns one map with KOut/VOut result values. As all initiated reduce commands return to a calling node, MapReduceTask simply combines all resulting maps into map M and returns M as a result of MapReduceTask.

Distributed reduce phase is turned on by using a MapReduceTask constructor specifying cache to use as input data for the task and boolean parameter distributeReducePhase set to true. Map/Reduce API javadoc and demos are included in distribution.

Moving forward

For Infinispan 5.2.0 final release we want to make sure the execution of intermediate migration key/value phase is as effective as possible and proven to be lock free for large input tasks as it was in our functional tests. We are also, as always, looking forward to your feedback and suggestions - especially if you have large data input sets ready for our latest MapReduceTask.


*If no keys are specified, entire cache key set will be used as in input.

Monday, 23 July 2012

Infinispan 5.2.0.ALPHA2 is here!

Infinispan 5.2.0.ALPHA2 was released last Friday with several additions for those that like to test Infinispan's bleeding edge capabilities. In this case, it's out Map/Reduce functionality that's the star of the show:
Vladimir Blagojevic, one of our Infinispan developers, will be explaining all about these features in a blog post coming right up, so stay tuned! :)

Finally Adrian Nistor, the latest addition to the Infinispan team, has been working on reducing the size of our distribution files by avoiding duplication of jars.


Wednesday, 11 July 2012

Infinispan's distributed executors and Map/Reduce in spotlight at JUDCon and JBW

JUDCon and JBoss World 2012 finished just a bit over a week ago in Boston and were a complete blast. Several of my colleagues presented their talks on the JBoss Data Grid and EAP clustering performance. However, JUDCon and JBoss World were particularly appealing to me personally as they made me aware of an increasing demand for large scale computing, and in particular, use of Infinispan's own distributed executors and Map/Reduce.

Anil Saldhana's talk about Big Data and Hadoop at JUDCon investigated the Hadoop setup in the JBoss ecosystem and its use for log analysis. In our discussion Anil contrasted the cumbersome Hadoop setup and API to our own Infinispan Map/Reduce solution.

Mark Addy's JUDCon "Infinispan from POC to Production" presentation was particularly engaging. Mark and his team at C2B2 developed a search engine for a UK based global on-line travel company using Infinspan as one of the key system components. One of their use cases involved extracting a particular pricing info from Infinispan cluster where distributed executors framework was an excellent fit. Long story short the response time improvement was an order of magnitude faster and the parallel execution on Infinispan cluster using distributed executors saved the day. 

Erik Salter's presentation "Infinispan == Profit: A Start-up’s Success with JBoss Community Software" summarized interesting details about video on demand service that Erik and his team developed for Cisco. Erik used the Infinispan cluster for session setup and management and found distributed executors and Map/Reduce to be a particularly good fit for a range of design trade offs he and his team faced. 

Stay tuned for more good things to come in this area!


Monday, 9 July 2012

JBoss Data Grid lands in Red Hat Summit!

It's just over a week since Red Hat Summit/JBoss World 2012 finished and it was a great pleasure to be part of it. Heiko Rupp and I were speaking about "Effectively Manage & Monitor Red Hat JBoss Data Grid Nodes" where we presented JDG and JON at a high level and then we showed a demo of both products interacting with each other. The presentation's slides are now available for download.

I was not the only one speaking about JBoss Data Grid. Both Manik, the Infinispan project lead, and Alan Santos, the product manager for Infinispan, were also delivering talks on JBoss Data Grid. Although their presentations are not up yet, you'll be able to download them from here.

I also met some Infinispan customers, such as Erik who's been using Infinispan at a well known telecommunications company, or two of the lead technical guys at a well known Geneva bank. We had some great conversations where we were able to synch up our roadmap with them to make sure any requirements they had are met in the future.

Of the presentations I attended, I was particularly impressed by Pete and Marius' 10 minute demo on going from 0 to a fully fledged mobile application running on top of AS7 in OpenShift in the "What's New in Java Frameworks for Web, Cloud, & Mobile" BOF. I hope it was recorded cos it was very impressive stuff.

It was a great week and once again it was a pleasure to be part of the Red Hat Summit and JBoss World :)


Friday, 22 June 2012

Infinispan 5.2 alpha out now with Command Line Inteface!

Infinispan 5.2.0.ALPHA1 has just been released and it comes with a load of new goodies, here's a summary:
Full details of all the rest enhancements can be found here. If you have feedback, please visit our forums. Finally, as always, you can download the release from here.


Infinispan CLI

When using Infinispan, both embedded as a library in an application or as a standalone server, I always longed for a simple standalone tool for interacting with the caches and the data within them. As all itches go, I had to scratch it, and so I present you with Infinispan's own CLI !

The CLI allows you to inspect and modify the data within an Infinispan cache and also provides access to some of the more advanced features (such as transactions).

The CLI is built out of two elements: a server-side module and the client command tool. The server-side module is optional, and provides the actual interpreter for the commands. Currently the server (and the client) use the JMX protocol to communicate, but in a future release we plan to support other communication protocols (in particular our own HotRod).

To get started, you need at least Infinispan 5.2.0.ALPHA1. Unzip the distribution and start a server:

./bin/ -r hotrod

The script automatically enables remote JMX connections and you can discover the port by running the jps command (part of the JDK/JRE) as follows:

jps -v

which should display something like

26532 Jps -Dapplication.home=/usr/lib/jvm/jdk1.7.0_04 -Xms8m
20508 Main"" -Dlog4j.configuration=file:////home/tst/Downloads/infinispan-5.2.0-SNAPSHOT/etc/log4j.xml

Now we can connect to the Infinispan instance using the CLI as follows:

./bin/ -c jmx://localhost:50434

You will be presented with a prompt:


The above prompt shows which host we're currently connected to and which CacheManager is being used (in this case: MyCacheManager).

Let's try putting some data in the cache

put a a

Now let's check that the cache actually contains the entry we've just put

get a

Which will display a glorious


The CLI understands several commands. Just type


to get a list of them and then

help [commandname]

to get help on a specific command's syntax.

The CLI uses the wonderful JReadline, so it supports all sorts of fancy buffer editing, history navigation and tab-completion as if you were in your comfortable OS shell (sorry Windows, cmd is not exactly a modern shell).

An important aspect of an Infinispan cache is that you can store whatever data you want in it. The CLI tries to interpret the data from the input you give it. It understands most of the Java native types (int, long, float, double, boolean, String), some additional fancy types (such as UUIDs) and a JSON syntax for mapping any type of Java class, so that you can write:

put user1 { "package.MyClass": {"i": 5, "x": null, "b": true } };

Conversely, when performing a get, the interpreter will output a JSON representation of your classes.

The CLI is still work in progress and will evolve and mature during Infinispan's 5.2 development cycle. You are all welcome to try it out and provide feedback on the forums, on IRC on channel #infinispan and using our issue tracker to report bugs and ask for enhancements.

I will soon be blogging again, hopefully with a video which will illustrate some of the more fanciful features of the CLI. Enjoy.

The JBoss Enterprise Data Grid tour!!

It's exciting times at the Infinispan team! Here's some of my highlights of the last few weeks and events coming up.

JBoss Enterprise Data Grid (based in Infinispan) presented in Spain's Red Hat JBoss Open Forum 2012

Yesterday I was in Madrid at Red Hat's JBoss Open Forum 2012, where I presented about JBoss Enterprise Data Grid and Clustering in JBoss Enterprise Application Plattform 6

This was a great opportunity to introduce these new products to a Spanish audience, and as with everything in Spain, if you present in the native language, people understand things better and are more likely to get a better impression. 

Might sound odd, but presenting this stuff in another language other than English, such as Spanish, is rather difficult even if you're a native speaker. If you've developed software in English, you're used to speaking about it in that language, but if you switch and you have to talk about in another language, getting the fluency required to tell a story (i.e. present something) takes a lot of preparation (write up notes in that language!) and above all, rehearsing.

The effort was worth it though! Having finished my first presentation, the JBoss Enterprise Data Grid one, I was non-stop talking to the Spanish sales force, potential Spanish customers...etc for the rest of the day, until I had to present again.

Last time I attended a JBoss middleware event in Spain was in Barcelona in 2006. A long time has passed since then, but what Red Hat Spain's JBoss Forum shows is that JBoss Middleware is back in the Iberian peninsula, and it's there to stay for a long time!!

Special thanks to Lucas Ponce who helped out massively with the live demos of both JBoss Enterprise Data Grid and JBoss Enterprise Application Plattform 6, and thanks to the rest of Red Hat Spain team who invited me to speak at the event.

JBoss World/Red Hat Summit and JUDCon in Boston

Heiko Rupp and I will be speaking about JBoss Enterprise Data Grid monitoring with JBoss Operations Network, which we're both hugely looking forward to! So, if you're around, don't miss the talk next Thursday at 2.30pm EDT (abstract can be found here)


Thursday, 21 June 2012

Fine-grained replication in Infinispan

Sometimes we have a large object, possibly with lots of attributes or holding some binary data, and we would like to tell Infinispan to replicate only certain part of the object across the cluster. Typically, we wanna replicate only that part which we've just updated. This is where DeltaAware and Delta interfaces come to play. By providing implementations of these interfaces we can define fine-grained replication. When we put some effort into such such an enhancements, we would also like to speed up object marshalling and unmarshalling. Therefore, we're going to define our own externalizers - to avoid slow default Java serialization.

The following code snippets are gathered in a complete example at This project contains a readme file with instructions on how to build and run the example. It is based on clustered-cache quickstart in Infinispan.

Implementing DeltaAware interface

So let's look at our main object. For the purpose of this exercise, I defined a Bicycle class that consists of many components like frame, fork, rearShock, etc. This object is stored in a cache as a value under certain (not important) key. It might happen in our scenario that we update only certain components of the bike and in such case we want to replicate just those component changes.

Important methods here are (description taken from javadocs):

commit() - Indicates that all deltas collected to date has been extracted (via a
                 call to delta()) and can be discarded. Often used as an optimization if
                 the delta isn't really needed, but the cleaning and resetting of       
                 internal state is desirable.

delta() - Extracts changes made to implementations, in an efficient format that
             can easily and cheaply be serialized and deserialized.  This method will
             only be called once for each changeset as it is assumed that any
             implementation's internal changelog is wiped and reset after generating
             and submitting the delta to the caller.
We also need to define setters and getters for our members. Setter methods are, among other things, responsible for registering changes to the changelog that will be later used to reconstruct the object's state. The externalizer for this class is only needed when cache stores are used. For the sake of simplicity, I don't mention it here.

Implementing Delta interface

Actual object that will be replicated across the cluster is the implementation of Delta interface. Let's look at the class. First, we need a field that will hold the changes - changeLog. Second, we need to define a merge() method. This method must be implemented so that Infinispan knows how to merge an existing object with incoming changes. The parameter of this method represents an object that is already stored in a cache, incoming changes will be applied to this object. We're using a reflection here to apply the changes to the actual object but it is not necessary. We could easily call setter methods. The advantage of using reflection is that we can set those fields in a loop.

Another piece is a registerComponentChange() method. This is called by an object of the Bicycle class - to record changes to that object. The name of this method is not important.

Defining our own externalizer

Alright, so what remains is the externalizer definition for the Delta implementation. We implement AdvancedExternalizer interface and say that only changeLog object should be marshalled and unmarshalled when transfering data over the wire. A complete (almost) implementation of Delta interface is the following.

Tell Infinispan about the extra externalizer

We also need to configure Infinispan to use our special externalizer to marshall/unmarshall our objects. We can do it e.g. programatically by calling .addAdvancedExternalizer() on the serialization configuration builder.

You can see we're also configuring transactions here. This is not necessary, though. We just aim to provide a richer example, removing transactional behavior is trully easy.

And here comes the "usage" part. Enclose cache calls by a transaction, retrieve a bicycle object from the cache, do some changes and commit them.

That's it. What is eventually transferred over the wire is just the changeLog object. The actual bicycle object is reconstructed from incomming updates.

If all of this seem to be too complex to you, I have good news. Infinispan provides one implementation of DeltaAware interface whish is called AtomicHashMap (package org.infinispan.atomic). If this map is used as a value in key/value pairs stored in the cache, only puts/gets/removes performed to this map during a transaction are replicated to other nodes. Classes like Bicycle and BicycleDelta are not need then. Even registering the externalizer for AtomicHashMap is not needed, this is done automatically during registration of internal externalizers. However, one might want a class emulating a real-world object, not just a map. That's the case when your own implementations of DeltaAware and Delta interfaces are the only way.

Wednesday, 20 June 2012

You can now buy support for Infinispan

Yes, at last.  Intentions first announced last year, and beta programme launched a few months ago, Red Hat's JBoss Data Grid (JDG) 6.0 is now in final, GA form.  This supported release is based on Infinispan 5.1.5.Final, and you can read more about it here.

It has been picked up by the press too.

Open source.  And enterprise-grade.  :-)


Thursday, 14 June 2012

Back from Berlinbuzzwords

I'm back from the berlin buzzwords conference where I've been talking about the datagrids performance. 
I've been told by several people how good this conference was - but even so I was very surprised by the quality of both the presenters and audience. The conference is very focused on data relates topics (scale, search, store) so it was great picking up the brains of so many data enthusiasts. The beer wasn't that bad either. 
A big thanks to the organizers for the this excellent event and hope that infinispan be part of next year's agenda.


Thursday, 31 May 2012

Infinispan 5.1.5 goes FINAL!

Infinispan 'Brahma' 5.1.5.FINAL has now been released fixing a whole bunch of issues around cache store preloading of distributed caches, Memcached server, tree module and Hot Rod client performance. We've also updated several libraries such as Netty (to 3.4.6) and JGroups (to 3.0.10).

Full details of what has been fixed can be found here, and if you have feedback, please visit our forums. Finally, as always, you can download the release from here.


Tuesday, 29 May 2012

Distributed execution framework enhancements

The initial release of the distributed execution included in Infinispan 5.0 was meant to be more of an experiment -- a testing ground -- rather than an industrial solution for distributed execution. We provided a simple interface similar to a familiar ExecutorService API and adapted it for execution on the Infinispan cluster. To our own surprise, use of the distributed execution framework took off and people started using it even in production environments. After the initial release users enthusiastically provided feedback and asked for enhancements. Some of these minor enhancements were integrated in subsequent Infinispan 5.1 and 5.2 releases. However, major improvements were put aside on a shelf, until now! The time is ripe to respond to these major enhancements so they can be ready for the Infinispan 6.0 release time. We have started a dedicated design page for distributed executor enhancements where you can provide your input and influence the design of the next iteration of the Infinispan distributed execution framework. Looking forward to more of your feedback!

Friday, 25 May 2012

On datagrid performance @ Berlinbuzzwords

I will be talking about datagrid performance and capacity planning and Radargun at this year's edition of Berlin Buzzwords, 4-5 June (I'll let you guess the location). My very first time at this conference but the amount of positive feedback I received about it was huge, so really looking forward to it!


Thursday, 24 May 2012

How to configure Infinispan with transactions, backed by relational DB on JBoss AS 7 vs. Tomcat 7

Migrating projects from one container to another is often problematic. Not as much with Infinispan. This article is about configuring Infinispan, using Transaction Manager for demarcating transaction boundaries, while keeping the data both in a memory and relational database - stored via JDBC cache store. I'll demonstrate all the features on code snippets. 

A complete application is located at and is called carmart-tx-jdbc. It's a web application based on JSF 2, Seam 3 and Infinispan 5.1.4.FINAL, is fully working, tested with JBoss  Application Server 7.1.1.Final and Tomcat 7.0.27. There  is one prerequisite, though. It needs an installed and working MySQL database in your system. The database name should be carmartdb, accessible by a user with carmart/carmart username/password.

First, look at what we need to configure for JBoss Application Server 7. 

Configuring transactions and JDBC cache store on JBoss AS 7

Infinispan will be configured via new fluent API using builders, hence the call to  .build() method at the end. We need to configure aspects related to  transactions and cache loaders. The configuration API for cache loaders  is likely going to be changed in not-so-far future. It should be fluent  and more intuitive, generally easier to use than current one. 

I purposely do not show XML configuration. Configuration examples can be found at In order to configure transactions and cache loaders, look for tags called  <transaction> and <loaders> and modify that sample file according to below configuration. Tag names and attribute names are very similar for both XML and Java configuration. If that is not enough, there is always a schema in Infinispan distribution.

The configuration of Infinispan is as follows: 

Lines marked with red are different in other containers/configurations, as you'll see in a minute. The code above implies that we need to specify proper TransactionManagerLookup implementation which is, in this case, GenericTransactionManagerLookup. We  also need to say: "Hey, I wanna use ManagedConnectionFactory as a connectionFactoryClass". OK, here we go. I should, as well, explain how to configure a datasource properly, right? In JBoss AS 7, this is configured as a subsystem in $JBOSS_HOME/standalone/configuration/standalone.xml:

The usage of transactions is very simple as we can obtain a transaction object by injection.


Quite easy, isn't it ...if you know how to do it. The only problem is that it does not work (at least not completely) :-) If you deploy the app, you find out that when storing a key-value pair in  the cache, an exception is thrown. This exception indicates that the operation with DB (and JDBC cache store) failed. The exception says:

A complete stack trace looks similar to
There's still an open issue in JIRA (ISPN-604) and it is being worked on. 

Configuring transactions and JDBC cache store on JBoss AS 7 - c3p0

But how do we cope with this inconvenience for now... By not using a managed datasource but rather a third party library called c3p0 (JDBC3  Connection and Statement Pooling, more information at Infinispan allows you to use this library for connecting to the database. If you really want to use it, you need to choose a different connectionFactoryClass which is, in this case, PooledConnectionFactory.

Infinispan configuration looks like this:

Transactions are accessible in the same way as in the previous use case. Now let's look at configuration for Tomcat servlet container. 

Configuring transactions and JDBC cache store on Tomcat 7

Tomcat does not have any Transaction Manager in it so we have to bundle one with the application. For the purpose of this exercise, we choose JBoss Transactions ( See dependencies at the end.

Cache manager and cache configuration is in this form:

For Tomcat, we need to specify a different transactionManagerLookup implementation and datasourceJndiLocation. Tomcat simply places objects  under a bit different JNDI locations. The datasource is defined in context.xml file which has to be on classpath. This file might look like this:

How do we get the transaction manager in the application then? Lets obtain  it directly from a cache. 

Infinispan knows how to find the manager and we need to know how to obtain it from Infinispan.

Sources: The transaction manager provides standard methods for transactions, such as begin(), commit() and rollback(). 

Now is the time for dependencies

So...which dependencies do we always need when using Infinispan with JDBC cache stores and transactions? These are infinspan-core, infinispan-cachestore-jdbc and javax.transaction.jta. The scope for jta dependency, as defined in Maven, is different for JBossAS and Tomcat.

Common dependencies for JBossAS and Tomcat

Of course, our application needs a few more dependencies but these are not directly related to Infinispan. Let's ignore them in this article. JBoss AS 7 provides managed datasource that is accessible from Infinispan. The only specific dependency (related to transactions or Infinispan) is JTA.

Dependencies specific to JBossAS - using managed Datasource (managed by the server)

Dependencies specific to JBossAS - using c3p0

Yes, you need to bundle also MySQL connector. On the other hand, for Tomcat use case and JBossAS with managed datasource, this jar file needs do be deployed to the server separately. For Tomcat, do this simply by copying the jar file to $TOMCAT_HOME/lib.  For JBoss AS 7, copy the jar file into $JBOSS_HOME/standalone/deployments.

Dependencies specific to Tomcat - using JBoss Transactions

That's it. I hope you've found this article helpful. Any feedback is welcome, especially the positive one :-) If you find any problem with the  application, feel free to comment here or participate in Infinispan forums (


Tuesday, 15 May 2012

One more micro release, Infinispan 5.1.5.CR1 out now!

Infinispan 5.1.5.CR1 has just been released with a handful of crucial issues that we considered required another community release. If you're a tree module user in JBoss AS7, or a an elastic Hot Rod server user,  or use XA datasources with Infinispan JDBC cache store, you should definitely upgrade to this version!

Infinispan 5.1 'Brahma' is a key component for a lot of projects both within and outside the boundaries of Red Hat, and this is reflected in all the micro releases we've done for 'Brahma' as Infinispan consumers near their final releases. From here I'd like to personally thank all the people behind these projects who have been crucial to the development of Infinispan. Expect a 5.1.5.FINAL in the next days which should wrap up the 'Brahma' release lifecycle barring a show-stopper! :)

As always, full details of what has been fixed can be found here, and if you have feedback, please visit our forums. Finally, as always, you can download the release from here.


Friday, 11 May 2012

Infinispan @ MOW 2012

A couple of weeks back I was speaking about Infinispan and Radargun at MOW 2012. Although this conference was originally focused on Oracle products, for a over a year now, Miracle have been expanding into other territories, such as Microsoft and Java middleware. However, there's still some work to be done in this area in order to make this a must-go conference for Java developers. Max Andersen and Thomas Heute were presenting as well on topics such as: Ceylon, JBoss AS7 and OpenShift.

If you want to find out more about Infinispan, don't miss Mircea Markus at Berlin Buzzwords 2012, at the beginning of June.


Thursday, 26 April 2012

Infinispan 5.1.4.FINAL is out now!

Infinispan 5.1.4.FINAL is out now fixing a bunch of issues as well as enhancing performance of Infinispan's Memcached and REST servers, plus it now exposes a couple of new JMX attributes such as cache view (RpcManager.CommittedViewAsString & RpcManager.PendingViewAsString) and cluster name.

Please note that we've changed the default value of worker threads for Infinispan servers to be twice the number of processors instead of 20 times the number of processors in order to be more efficient out of the box. The number of worker threads is of course configurable, so you can always modify it according to your use case.

Full details of what has been fixed can be found here, and if you have feedback, please visit our forums. Finally, as always, you can download the release from here.


Tuesday, 24 April 2012

GSoC and Infinispan

I am delighted to announce that one of our submissions for GSoC was accepted! Sunimal Rathnayake will join the infinispan team for implementing a HotRod client in C#. This would allow Infinispan clusters to be consumed by .NET clients which something we've planned adding for a long time.

Welcome onboard Sunimal and looking forward to working with you!

Monday, 23 April 2012

Back from Portugal with goodies

The trip started in Coimbra where Sanne Grinovero and I discussed about Infinispan and OGM at the Portugal JUG. The interest was obvious from the amount of questions and the discussions that followed and ended up late in the night. A big thanks to Samuel Santos for organising this!
The trip then continued in Lisbon where we meet the CloudTM team for two good days of hacking and brainstorming.  The results are a set of awesome features the CloudTM is about to contribute to Infinispan, especially around transactions:
  • Total Order Broadcast (TOB) - is a transaction protocol for replicated caches built on atomic broadcast. It relies on JGroups'  SEQUENCER protocol for achieving total order. The benchmarks ran comparing it with the current 2PC based transaction protocol are rather promising and we hope to integrated it in our 5.2 release
  • Total Order Multicast (TOM) - is simplistically put TOB for distributed caches. This is the next in pipe after TOB to be integrated. Again, preliminary performance comparisons look rather promising
  • The  CloudTM also extended Infinispan's transactions to support one-copy serializability (that's a fancy name for SERIALIZABLE  transactions in replicated systems). The implementation is based on keeping multiple data version of data for each entry together with some housekeeping code. More on this to come!
Big thanks to CloudTM team for their effort and all the great ideas (and patches!!) they submitted to Infinispan! It was a great meeting with very good results and burning whiteboards (see below)!

Friday, 13 April 2012

Introducing the JBoss Data Grid: Infinispan, with support!

To many who are familiar with Red Hat's model of unsupported upstream projects with supported, heavily tested and certified controlled-release "products", the announcement of the JBoss Data Grid (JDG) will come as no surprise.  JDG is to Infinispan what Red Hat Enterprise Linux is to Fedora, or what JBoss Enterprise Application Platform (EAP) is to JBoss AS.  Folks considering deploying Infinispan in a mission-critical production environment should consider JDG instead, not only to gain the benefits of a more thorough quality control and certification process but also to allow Red Hat to provide development and production support and consultancy.

JDG was announced at Red Hat Summit/JBoss World last year, and I blogged about it here, and now JDG has reached a stage where it is available as a public beta with a GA release coming soon.  If you are interested in JDG, or supportable Infinispan, I encourage you to register your interest in the JDG Beta.


Infinispan 5.1.4.CR1 is here!

Infinispan 5.1.4.CR1 is out now with minor improvements focusing on third party library upgrades such as JBoss Transactions and JGroups, and state transfer related issues, and reducing the resource consumption of testsuite.

Full details of what has been fixed can be found here, and if you have feedback, please visit our forums. Finally, as always, you can download the release from here.


Infinispan and OGM at PT.JUG

Sanne Grinovero and myself will talk about Infinispan and OGM at Portugal JUG in Coimbra on the April 18. This join talk covers an introduction to Infinispan and its main use cases followed by a presentation about NoSQL and Hibernate OGM. Thanks to Samuel Santos for inviting us and really looking forward to meet with the community!