Tuesday, 14 November 2017

Merci Duchess et Devoxx!

We've had two very hectic weeks delivering the Infinispan/Vert.x/OpenShift Streaming Data workshop in Duchess France and Devoxx Belgium. First of all, thanks to all attendees for taking the time to attend this workshop and working through it, thanks Duchess and Devoxx for letting us present the workshop, and thanks the sponsors for making it happen!

In case you want to walk through the workshop at your own pace, the version of the workshop delivered at Duchess and Devoxx can be found here. The slides from the workshop can be found here.

The aim of the workshop was to show how to work with multiple data streams and combine them in interested ways. These two streams of data came from Transport API at OpenData.ch and SBB.ch respectively. The end-game of the workshop was to see to combine both sets of information to produce a map that tracked positions of delayed trains around Switzerland.

On 2nd November we did a dry run of the workshop in the Duchess France group. This was a very compact group of people, around 10-12, which gave us thve first taste of what it was like to work through it. In particular, we realised that getting Virtual Box to run on each attendees laptop took some time to get working. From the workshop perspective, we found that the exercises themselves were a bit too long for the people to complete.


With all the invaluable input, Katia, Thomas and myself got ready for delivering the workshop at Devoxx Belgium, one of the top developer conferences in Europe. The pressure was on! We had ~60 attendees, so we had to make sure everyone could progress.



At Devoxx, we were more prepared to help out with set up problems, so everyone got to the starting point much faster. After working through the initial exercise to get used to Infinispan, Vert.x and OpenShift, attendees got on with the workshop itself.

During the workshop exercises, the learnt about Infinispan's in-memory data grid capabilities, and in particular about continuous query which is very well suited for feeding data to live updating maps or dashboards. They also learnt about how easy it is to build reactive applications with Vert.x and RxJava. Finally, they learnt how to deploy applications to OpenShift, how to monitor their progress...etc.

A majority of Devoxx attendees managed to get to the end of the workshop which was a success for us, but we listen to their feedback and we will continue to improve the content and delivery as we prepare for Codemotion Madrid where we are delivering the workshop once again.

Thanks attendees, Devoxx, Duchess and sponsors!!

Cheers,
Galder

Infinispan 9.2.0.Beta1 and 9.1.3.Final have been released

Dear Infinispan Community,

Two new releases are ready for you today, our newest and greatest 9.2.0.Beta1 and an updated Final of our stable branch, 9.1.3. Both can be found as usual on our download page.

The highlights of 9.2.0.Beta1 are:

  • New API for querying using Ickle via the REST server, taking advantage of the JSON/Protobuf interoperability [ISPN-8113].
  • Clustered Locks, first experimental release. A ClusteredLock is a data structure used for concurrent programming between Infinispan instances in cluster mode.
  • Clustered Counters have received serveral improvements. It's now possible to manage your counters via JMX [ISPN-7926] and other management methods are on the way. The strong counter has a new method: compare-and-swap [ISPN-8489]. If you used compare-and-set, the compare-and-swap may improve your code performance. And last but not least, a new sync() method was added to the strong and weak counters, useful if you don't want to handle the CompletableFuture!
  • Clustered Cache Configuration: Both EmbeddedCacheManager and RemoteCacheManager now have an API for creating/removing caches across the cluster [ISPN-7776]. Caches created with this API will also be created on any new members that will join the cluster. Additionally, the configuration for caches created in this way can be made persistent so that they are automatically recreated when a cluster restarts.
  • Server deployment of Lucene analyzers for remote query [ISPN-7714].

9.1.3.Final and 9.2.0.Beta1 both add several Off-heap improvements and fixes.

Full details of the new features and enhancements included in these two releases can be found here and here.

We have even more exciting stuff coming up for Infinispan 9.2. Thank you for following us and stay tuned!

The Infinispan Team

Tuesday, 31 October 2017

Infinispan coming to Duchess France and Devoxx Belgium !




This week, 2nd November, Galder and I will be presenting at Duchess France Meetup in Paris our "Streaming Data Workshop" : a 3h hours hands on lab that showcases a slightly simplified streaming architecture built on top of Infinispan and Vert.x and running on OpenShift

If you are in Paris, the Duchess France workshop is actually full; if you are planning to assist Devoxx Belgium this year, the workshop will take place the 6th of November there.


Cheers,
Katia 

Friday, 27 October 2017

Infinispan 9.2.0.Alpha2 & 9.1.2.Final released

Dear Infinispan Community,

We have 2 new releases ready for everyone. We have our newest and greatest, 9.2.0.Alpha2 and an updated Final for our stable branch 9.1.2. As usual, both can be found on our download page.

9.2.0.Alpha2 contain the following new features

  • New Counters can be removed ISPN-8093
  • Interoperability between JSON and Protobuf, allowing to retrieve Protobuf content (written via Hot Rod) as JSON via Rest; also JSON documents written via Rest can be automatically converted to Protobuf and indexed. ISPN-7422
  • Wildcard cache configurations allowing for multiple caches from 1 config ISPN-8379
  • Reactive streams based iterator implementation providing better throughput and less resources usage ISPN-7865
  • An experimental evalAll like method on the LockedStream interface allowing for exclusive operation on entries and a return value ISPN-8310

9.1.2.Final and  9.2.0.Alpha both added

  • Updates to third party JGroups, Hibernate and Hibernate Search
  • Off heap improvements and fixes for Query & Encoding

Besides these there were also quite a few bug fixes and code clean ups. Full details of the new features and enhancements included in this release can be found here and here.

We have even more exciting features still coming up for Infinispan 9.2. Thank you for following us and stay tuned!

The Infinispan Team

Monday, 23 October 2017

Thanks Basel One 2017!

Thanks a lot to all attendees at Basel One 2017, it was great fun presenting about Infinispan, OpenShift and Vert.x topics just around the corner from my house :)

The slides for my talk can be found here. The code and scripts demo that I showed can be found here. I've also uploaded step-by-step instructions for the live-coding done during the talk.



Cheers,
Galder

Friday, 20 October 2017

Cache Store Batch Operations

Infinispan 9.1.x introduces batch write and delete operations for cache stores. The introduction of batching should greatly improve performance when utilising write-behind cache stores, using putAll operations and committing transactions in non-transactional stores.


CacheWriter Interface Additions


The CacheWriter interface has been extended so that it exposes two additional methods: deleteBatch and writeBatch.  For the sake of backwards compatibility a default implementation of these methods is provided, however if your cache store is able to utilise batching we strongly recommend you create your  own implementations. The additional methods and docs are show below: 



Updated Stores


Currently the JDBC, JPA, RocksDB and Remote stores have all been modified to take advantage of these latest changes.


Configuration Changes


As each store implementations has different batching capabilities, it was necessary to introduce a max-batch-size attribute to the AbstractStoreConfiguration. This attribute defines the maximum number of entries that should be included in a single batch operation to the store. If a value less than one is provided, then the underlying store implementation should not place a upper limit on the number of entries in a batch. 

Deprecated Attributes

Both TableManipulationConfiguration#batchSize and JpaStoreConfiguration#batchSize have been deprecated, as they serve the same purpose as AbstractStoreConfiguration#maxBatchSize.

Store Benchmark


To measure the impact of batch writes on Cache.putAll, we created a simple benchmark to compare the performance of Infinispan 9.1.1.Final (with batching) and 9.0.3.Final (without).  The benchmark consisted of 20 threads inserting 100000 cache entries as fast as possible into a cache via putAll; with each putAll operation containing 20 cache entries and the max-batch-size of each store being set to 20. The table below shows the average time taken for each store type after the benchmark was executed three times.


Store Type 9.0.3.Final 9.1.1-Final Latency Decrease
JdbcStringBasedStore 29368ms 2597ms 91.12%
JPAStore 30798ms 16640ms 45.97%
RocksDBStore 1164ms 209ms 82.04%

The benchmark results above clearly show that performance is increased dramatically when utilising batch updates at the store level.


Conclusions


Infinispan 9.1.x introduces batching capabilities to the CacheWriter interface in order to improve performance. If you currently utilise a custom cache store, we strongly recommend that you provide your own implementation of the delete and write batch methods. 

If you have any feedback on the CacheWriter changes, or would like to request some new features/optimisations, let us know via the forumissue tracker or the #infinispan channel on Freenode.

Wednesday, 18 October 2017

Infinispan coming to Basel One 2017!


Tomorrow, 19th October, I'll be presenting at Basel One 2017 a talk on "Streaming Data Analysis with Kubernetes", where I'll be combining OpenShift, Infinispan and Vert.x to demonstrate how to process streaming data.

If you're in Basel and want to find out more about these technologies, make sure you come! The talk is at 16:00 :D

Cheers,
Galder