Tuesday, 11 December 2018

Infinispan Spring Boot Starter 2.1.1.Final and 1.0.4.Final are out!

Dear Infinispan and Spring Boot users,

We have just released Infinispan Spring Boot 2.1.1.Final and 1.0.4.Final.

1.0.4.Final is using Spring Boot 1.5.17.RELEASE
2.1.1.Final is using Spring Boot 2.1.1.RELEASE 

Both starters are now using Infinispan's last stable release: 9.4.4.Final.

9.4.4.Final provides full support to JDK 11, Spring 4 and 5.
Concerning Spring 5, it includes Spring Session 2.1.2.RELEASE support.

You can find these releases in the maven central repository.

Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Monday, 10 December 2018

Infinispan 9.4.4.Final and 10.0.0.Alpha2 out with Spring 5 support!

We've just released completed a couple of releases:

Infinispan 9.4.4.Final (release notes)

  • Adds Spring 5 and Spring Boot 2.1 support
  • Fixes for JDK 11 compatibility. 
  • Improves Cache#size() performance when using shared cache stores.

Infinispan 10.0.0.Alpha2 (release notes) contains same fixes as 9.4.4.Final plus

  • Removes the CLI cache loader
  • RemoteCache#getBulk methods.

Don't forget to checkout the Infinispan 10 road map blog post to find out all that's coming.

Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Cheers,
Galder

Wednesday, 21 November 2018

The road to Infinispan 10 (Alpha1)

Dear all,

Today we are releasing 10.0.0.Alpha1 and 9.4.2.Final.

Infinispan 9.4.2.Final comes with a number of bug fixes and some small additional features:

  • ISPN-9655 REST Access Log headers
  • ISPN-8144 & ISPN-9661 Cross-Site replication statistics
  • ISPN-9708 Expose the executor services through JMX
  • ISPN-9732 Local iteration optimization with write behind is valid for non shared stores
  • ISPN-9717 Fix Integer overflow for lifespan and maxIdle

We have begun working on what will become Infinispan 10. As with all new major releases, this will come with a number of important changes.

  • New Server
    We are working on a new lightweight server, currently dubbed ServerNG, which will supersede the current WildFly-based offering. The new server will have a smaller disk and memory footprint, a new RESTful admin interface, improved security. It will still use familiar components (Elytron for security, Narayana for transactions, etc) but we hope that the installation and usability experience will be most improved. A dedicated blog post will describe in detail what is coming.
  • Long-term Storage Format
    The persistent storage format will be changed so that it will be easier to transparently make changes to it without requiring further exporters/importers.
  • Non-blocking listeners
    The listener implementation will be replaced with a non-blocking implementation.
  • Asynchronous CacheLoader/Store
    Store operations will be ran in another thread to provide non blocking for main threads
  • Improved statistics
    Infinispan statistics have been traditionally over-simplistic, offering mostly basic averages for writes and reads. We are going to implement percentiles on a histogram as well as recording tracing information so that you will be able to know how much time is being spent in the various subsystems (clustering, persistence, etc.)
  • New API
    The current Infinispan API, based around Java's ConcurrentHashMap design, does not offer the flexibility required to support modern reactive designs as well as the various extensions we've added over the years (counters, multimaps, etc). We are therefore working on a new modern API design which we will be describing with a number of blog posts in the near future.
  • Agroal JDBC Connection Pool
    We are replacing the JDBC connection pool implementation with Agroal.
  • Kubernetes Operators
    Operators are all the rage in the Kubernetes world, and we are working on an Infinispan Operator which will take care of managing and monitoring the health of an Infinispan cluster, handle scale up/scale down safely, perform upgrades and more.

Infinispan 10.0.0.Alpha1 is the first release from our development branch. It currently includes the following features on top of what is in 9.4.2.Final:



Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Monday, 19 November 2018

Quick start Infinispan on Kubernetes

Last week we showed you how to easily run Infinispan on top of OpenShift. This week we're trying to do the same on Minikube, a tool that makes it easy to run vanilla Kubernetes locally.

Although we've already covered the topic in the past, we felt the descriptors needed a permanent location and an update to the latest Infinispan releases. Detailed instructions can be found in this repository.

With OpenShift, we took advantage of Templates which allow a set of objects to be parameterised.
Templates are OpenShift specific, so Kubernetes does not understand them. Instead, we provide you with the individual descriptors required to run Infinispan (Helm chart to come...). This includes:


Before applying the descriptors, download and install Minikube. Then, set a profile, select the VM driver, give it enough CPU and memory for your experiments, and start it.

Once Minikube it's running and you have the corresponding kubectl command line tool installed, simply call:

$ kubectl apply -f .

Once all pods are ready, you should verify the 3-node cluster has formed correctly (find out how in the README file).

When ready, you can start storing and retrieving data. The HTTP REST endpoint is particularly useful for these initial tests, to verify everything works as expected:

$ kubectl exec \
  -it infinispan-server-0 \
  -- curl -v -u test:changeme -H 'Content-type: text/plain' -d 'test' infinispan-server-http:8080/rest/default/stuff

Then:

$ kubectl exec -it infinispan-server-1 \
  -- curl -v -u test:changeme infinispan-server-http:8080/rest/default/stuff

Go and try it out and let us know what you think. You can find us on this Zulip chat :)

Cheers,
Galder

Thursday, 15 November 2018

Hotrod clients C++ and C# 8.3.0.Final are out!

Dear Infinispanners,

The C++ and C# 8.3.0.Final releases are out!

Main features contained in this release are:
  • Cache Admin Operations: create and remove cache at runtime;
  • Counters: clusterwide counters;
  • Transactions: run a list of operations transactionally;
  • Media Types: use differents media-types to encode (key,value) pairs.
Source code, binaries and docs are available as usual at the links below.

Thank you for reading,
The Infinispan Team


[1] Release notes for the 8.3.0 serie
[2++] C++ code for 8.3.0.Final
[2#] C# code for 8.3.0.Final
[3] Downloads

Monday, 12 November 2018

The fastest path to running Infinispan on OpenShift!

Creating an Infinispan Server cluster in OpenShift has never been easier! We've just given the OpenShift templates for Infinispan server their biggest makeover yet which should help both Infinispan and OpenShift users:

The repository has been simplified and flattened out to only leave essential information. Minishift is the preferred way to get started with Infinispan and OpenShift, so we've tailored the instructions for this set up.

OpenShift templates are now YAML based which is less verbose, but more importantly, allows Infinispan Server XML configuration to be shown as-is. This makes it easier to directly modify the XML in the template itself.

The fastest way to get started with Infinispan and OpenShift is to simply fire up Minishift, set a profile, checkout our Infinipan OpenShift repository and then call:

oc create -f infinispan-ephemeral.yaml
oc new-app infinispan-ephemeral

These two simple steps will get you a single node Infinispan Server running! A more detailed getting started guide can be found in the repository's README file.

Go and try it out and let us know what you think. You can find us on this Zulip chat :)

Cheers,
Galder

Friday, 9 November 2018

Infinispan 9.4.1.Final and Infinispan Spring Boot Starter 2.1.0.Final are out!

Dear Infinispan and Spring Boot users,

We have just released Infinispan 9.4.1.Final and Infinispan Spring Boot 2.1.0.Final.

Highlights of the Infinispan release include:

Complete release notes can be read here.

Highlights of the Infinispan-Spring-Boot release include:
  • Upgrade Spring-Boot version to 2.1.0
  • Upgrade Infinispan version to 9.4.1
  • Integration with Spring Actuator, to expose production ready metrics (ISPN-9668)
  • Bug fixes
  • Additional code examples
You can find these releases in the maven central repository.

Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Friday, 2 November 2018

Infinispan triple connector release!

Infinispan Spark, Infinispan Hadoop and Infinispan Kafka have a new fresh release each!


Infinispan Spark 0.9

 

The native Apache Spark connector now supports Infinispan 9.4.x and Spark 2.3.2, and it exposes Infinispan's new transcoding capabilities, enabling the InfinispanRDD and InfinispanDStream to operate with multiple data formats. For more details see the documentation.


Infinispan Hadoop 0.3

The connector that allows accessing Infinispan using standard Input/OutputFormat interfaces now offers compatibility with Infinispan 9.4.x and has been certified to run with the Hadoop 3.1.1 runtime. For more details about this connector, see the user manual. Also make sure to check the docker based demos: Infinispan + Yarn and Infinispan + Apache Flink.


Infinispan Kafka 0.3

 

Last but not least, the Infinispan Kafka connector was upgraded to work with the latest Kafka (2.0.x) and Infinispan releases (9.4.x). Many thanks to Andrea Cosentino for contributing this integration.



Infinispan 9.3.4.Final released!

This is a bug fix release and users currently on 9.3.x are encouraged to upgrade, specially when using the query functionality.

For a list of fixes, please consult the release notes.


Near caching with Spring-Boot and Infinispan


We have recently released infinispan-spring-boot-starter 2.0.0.Final. This version supports Spring Boot 2.1 and Infinispan 9.4.0.Final.

Before this release, some important features - such as near caching - were only configurable by code.
From now on, we can set all of the Hot Rod client configuration using the hotrod.properties file or the Spring application YAML. The latter is an important community requirement we had.

Let's see how to speed up our applications performance with near caching!


Hot Rod

 

Just as a quick reminder, Infinispan can be used embedded in your application or in client/server mode. To connect you application to a server you can use an Infinispan Client and the Infinispan “Hot Rod Protocol”. Other protocols are available, such as REST, but Hot Rod is the most recommended way since it is the one that supports most of the Infinispan functionalities.

Near cache


From the Infinispan documentation: Hot Rod client can keep a local cache that stores recently used data. Enabling near caching can significantly improve the performance of read operations get and getVersioned since data can potentially be located locally within the Hot Rod client instead of having to go remote.

When should I use it? 


Near caching can improve the performance of an application when most of the accesses to a given cache are read-only and the accessed dataset is relatively small.
When an application is doing lots of writes to a cache, invalidations, evictions and updates to the near cache need to happen. In this scenario we won't probably get much benefit.

As I said in the introduction, the good news is that this feature can be activated just by configuration. Code doesn't change, so we can measure the benefits, if such exist, in a very straightforward way.

Spring-Boot


I have created a very simple application, available here. Maven, Java 8 and an Infinispan server are required to run it. You can download the server or use docker.


Docker: docker run -it -p 11222:11222 jboss/infinispan-server:9.4.0.Final

Standalone: PATH/infinispan-server-9.4.0.Final/bin/standalone.sh

Once the server is up and running, build the application using maven 

>> infinispan-near-cache: mvn clean install

Writer 


This application loads the required data to a remote cache: a list of some of the Infinispan contributors over the last decade.

>> writer: mvn spring-boot:run


Reader 


The reader application does 10.000 accesses to the contributors cache. Using a random id, I call 10.000 times the get method. The job gets done in my laptop in ~4000 milliseconds.

>> reader-no-near-cache: mvn spring-boot:run


Activating the near cache


I need to configure two properties:
  • Near Cache Mode: DISABLED or INVALIDATED. Default value is DISABLED, so I turn it on with INVALIDATED.
  • Max Entries: Integer value that sets the max size of the near caches. There is no default value, so I set up one.
The hotrod client configuration is for each client, not for each cache (this might change in the future). With that in mind, note that configuring the previous properties will activate near caching for all the caches. If you need to activate it just for some of them, add the following property:
  • Cache Name Pattern:  String pattern. For example "i8n-.*" will activate the near caching for all the caches whose name starts by "i8n-".

Configuration can be placed in the hotrod-client.properties, Spring-boot configuration or code.

hotrod-client.properties

infinispan.client.hotrod.near_cache.mode=INVALIDATED
infinispan.client.hotrod.near_cache.max_entries=40
infinispan.client.hotrod.near_cache.cache_name_pattern=i8n-.*

application.yaml (or properties)

infinispan:
   remote:
     near-cache-mode: INVALIDATED
     near-cache-max-entries: 10
     near-cache-cache-name-pattern: i8n-.*

code 

With the Infinispan Spring-Boot Starter, I can add custom configuration using the InfinispanRemoteCacheCustomizer.


Results


My dataset contains 25 contributors. If I activate the near cache with max 12 entries and I run my reader again, I get the job done in ~1900 milliseconds, which is already an improvement. If I configure it to hold the complete dataset, I get it done in ~220 milliseconds, which is a big one!

Conclusions


Near caching can help us speed up our client applications if configured properly. We can test our tuning easily because we only need to add some configuration to the client. Finally, the Spring-Boot Infinispan Starter helps us build services with Spring-Boot and Infinispan. 

Further work will be done to help Spring-Boot users work with Infinispan, so stay tuned! Any feedback on the starter or any requirement from the community is more that welcome. Find us in Zulip Chat for direct contact or post your questions in StackOverflow!




Wednesday, 31 October 2018

Native JSON and Node 8.11 baseline in Node.js client 0.6.0!!

Infinispan Node.js client 0.6.0 is out now!! :)

As well as updating the client so that it understand the latest Hot Rod protocols in Infinispan 9.4.0.Final, this version comes with native JSON object support.

To make the Node.js client backwards compatible, the client still treats key/value pairs as String by default. If you want to use native JSON objects, you have to explicitly configure the Node.js client to do so (see example).

Starting with this version, we've upgraded the base Node version requirement to 8.11, which is the latest stable release branch at the time of writing. With such upgrade, the client no longer needs to use external promise dependency which was know to leak.

If you're a Node.js user and want to store data remotely in Infinispan server instances, please give the client a go and tell us what you think of it via our forum, via our issue tracker or via Zulip on Infinispan channel.

Cheers
Galder

Thursday, 11 October 2018

Infinispan Spring Boot 2.0.0.Final is out!

Dear Infinispan and Spring Boot users,

We have just released Infinispan Spring Boot 2.0.0.Final.
If you are wondering why it is worth to use this starter, read Sebastian's article here!

Highlights of this release include:
  • Uses the latest Infinispan 9.4.0.Final
  • Automatic translation of Hot Rod client properties into Spring YAML (ISPN-9437)
  • Bug fixes

You can find the release in the maven central repository.

Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Monday, 8 October 2018

Infinispan 9.4.0.Final

Infinispan 9.4.0.Final “Infinity Minus ONE +2”

 

10 years of Infinispan

Infinispan is 10 years old this month, and what better way to celebrate than with a brand new Final release !!!

What's new

Infinispan 9.4 comes with the following new features / improvements:
  • Segments everywhere
    9.3 brought the segmented on-heap memory container. 9.4 extends this to provide the benefits of segmentation to off-heap as well as all of the core cache stores. Watch as your bulk operations (size, iteration, streams) get a big performance boost !
  • Transcoding everywhere
    To paraphrase the Grand Moff Tarkin, “The last remnants of Compatibility Mode have been swept away”. Transcoding, i.e. the ability to transparently convert between a number of formats across different endpoints, is now “fully operational”.
  • Transactions everywhere
    Hot Rod transactions now support recovery.
  • Hot Rod client improvements
    The Hot Rod client has received many improvements:
    • Client-side statistics, complete with JMX support
    • Improvements to the scalability and the behaviour of near-caches
    • All of the configuration can now be supplied via the properties, which also means easier integration with other frameworks, such as Spring Boot.
  • Query improvements
    Many cleanups and improvements
  • Bugfixes, stability, reliability
    Although not as exciting as new features, we continued our work to improve the stability, reliability and performance of all aspects of Infinispan.
  • Upgrades:
    • As usual the latest and greatest JGroups 4.0.15
    • The server is now based on WildFly 14

Get it, Use it, Ask us!

Please download, report bugs, chat with us, ask questions on the forum or on StackOverflow.

Tuesday, 2 October 2018

Segmented Data Containers: Distributed Stream Performance Boost

Welcome to the first of several blog posts that describe the segmentation of containers that Infinispan uses to store data. Some of you may have noticed in the previous 9.3.0.Final notes that we announced a new feature named “Segmented On-Heap Data Container”. We also mentioned that “It improves performance of stream operations”, but what does that really mean?

What is a segmented data container and why does it matter? 


Imagine a cluster of 4 nodes in distributed mode (numOwners = 2) with entries for k0 - k13. It might look like this:




The data is distributed between the nodes with only two copies of each entry available. However, the data itself is stored internally in the same Map instance. As a result, when performing operations on all entries in the cache, Infinispan must iterate over the same data multiple times. This degrades performance.

As of Infinispan 9.3, a segmented data container is available to separate data by segments. Although only on-heap bounded and unbounded implementations are currently available.

With a segmented data container, that same data set might look like this:




Because Infinispan internally reasons on data in terms of segments, a segmented data container lets Infinispan process data only in specific segments. This allows for operations performed upon all entries to require iteration over the data only once.

Actual Performance Difference


So with the above example you might be thinking that the performance increase maximum is two times throughput, since numOwners is two. This is close, but not quite correct. While iterating on the data we also have to determine what segment an entry belongs to. With a segmented container we know this already, so there is no need to calculate that. This provides additional performance, as you will see.

The following graphs were generated using the benchmark at https://github.com/infinispan/infinispan-benchmarks/tree/master/iteration. The following command was run: java -jar target/benchmarks.jar -pvalueObjectSize=1000 -pentryAmount=50000 -pbatchSize=4096



The preceding graph is the result of the iteration methods. As you can notice the performance increase isn’t that much… why not?!?

Unfortunately, remote iteration requires a lot of network overhead, so we don’t get to see the full benefits of segmentation. But at least it is about 5-12% faster, not too shabby.

Now to show the real improvement, here is the chart showing the performance increase for the Cache#size operation:



If you notice there is huge increase in performance: almost a three fold increase over the non-segmented container, even though numOwners is only two. The old segment calculation adds a bit of overhead compared to just incrementing a number.

So keep in mind this change will show a larger gain in performance if the result returned is smaller, especially if it is a fixed size, such as a single int for Cache#size.

What about gets and puts?


Having the container segmented should also affect get and put performance as well, right? In testing the difference for get and puts are less than one percent, in favor of segmentation due to some optimizations we were able to add.

How do I enable this?


So the performance gains are noticeable, especially when the remote operation returns a small data set. But how can a user configure this? This is the nice part, due to no performance loss with other operations the container will always be segmented as long as the cache mode supports segmentation. That is if it is a Distributed, Replicated or Scattered cache.

A real-life example and closing


Since this feature has been around a while already, we actually have users gaining benefits from this feature. An example can be found at https://developer.jboss.org/message/983837#983837. In this case the user only upgraded to Infinispan 9.3 and received over a three-fold increase in performance when using distributed streams. It actually starts to bring distributed streams performance within range of indexed query for some use cases.

So, by upgrading your application to Infinispan 9.3 or newer, you will benefit from these improvements. There will be future posts regarding segmentation, including support for stores. Either way please feel free to download Infinispan, report bugs, chat with us, ask questions on the forum or on StackOverflow.

Monday, 17 September 2018

Infinispan 9.4.0.CR3, 9.3.3 and codename vote

Hi Infinispan Community,

our original plan was to release 9.4 Final today, but we have decided to delay the release by an extra sprint (3 weeks).
So today we are announcing 9.4.0.CR3 instead, which comes with the following fixes:

  • Segmented off-heap data container
  • Performance improvements for Near Caches
  • Hot Rod client-side statistics
  • Removal of the old compatibility mode in favor of the new transcoding capabilities is now complete
  • Server rebased on top of WildFly 14
  • Lots of code cleanups, especially around query
  • Bug fixes
For those of you on our stable branch, we also released a 9.3.3 with some bug fixes.
Please download, report bugs, chat with us, ask questions on the forum or on StackOverflow.

As this release is yet unnamed, please make sure you vote for the name:

Tuesday, 4 September 2018

Infinispan Spring Boot Beta2 is out!

Dear Infinispan and Spring Boot users,

We have just released Infinispan Spring Boot 2.0.0.Beta2.

Highlights of this release include:
  • Upgrade to Spring Boot 2.0.3.RELEASE
  • RemoteCache can be injected now. Example here 

You can find the release in the maven central repository.

Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Thursday, 30 August 2018

Infinispan 9.3.2.Final and 9.4.0.CR2 are out!

Dear Infinispan users,

We have just released 9.3.2.Final and 9.4.0.CR2.

9.3.2.Final includes bug fixes and improvements.

Highlights of this release include:
  • Upgrade to protostream 4.2.1.Final (ISPN-9399
  • Ship basic analyzers by default (ISPN-9429
  • Avoid unneeded parentheses in Ickle query string generated by QueryBuilder (ISPN-9378
The full list of 9.3.2.Final fixes are here.

9.4.0.CR2 has been released due to an internal issue we experienced releasing CR1.

You can find both releases on our download page. Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Monday, 27 August 2018

Infinispan 9.4.0.CR1 is out!

Dear Infinispan users,

We have just released 9.4.0.CR1 which includes bug fixes and improvements.

Highlights of this release include:
  • Near cache and client side TTL (ISPN-9098
  • Allow configuring near cache from hot rod properties (ISPN-9385
  • Initial support for JSON format in scripts 
  • Default analyzers are provided to use with remote queries (ISPN-9429
  • Prevent user from configuring passivation with a shared store (ISPN-7168
  • Enhance RemoteStore to support segmentation (ISPN-9376
  • Upgrade to RxJava 2.2.0 (ISPN-9418
  • Hot Rod configuration properties have setters now to ease integration with other frameworks (ISPN-9398
The full list of 9.4.0.CR1 fixes are here.

You can find the release on our download page. Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,

The Infinispan Team

Tuesday, 14 August 2018

Hotrod clients C++ and C# 8.3.0.Beta1 are out!

Dear Infinispanners,

The C++ and C# 8.3.0.Beta1 releases are available!

Main feature for this release is: transactions. Clients can now run sequence of hotrod operations in a transactional way. Basic methods are provided to begin, commit or rollback a transaction over an hotrod connection (hotrod 2.7 and Infinispan 9.3+ are required).
API are quite easy to use:
Source code, binaries and docs are at the usual place. Thank you for following us!
The Infinispan Team
[1] Release notes
[2++] C++ code for 8.3.0.Beta1
[2#] C# code for 8.3.0.Beta1
[3] Downloads

Monday, 13 August 2018

Node.js client 0.5.0 released with improved stability and better OSX integration

Infinispan Node.js client 0.5.0 was released last week. It comes with much improved stability under heavy load conditions and hence it's a recommended upgrade for any current users.

On top of that, a configuration option called topologyUpdates (true (default) / false) has been added to disable topology updates. This can be useful when trying to access Infinispan server running within a Docker container on MacOs. Without this option to disable topology updates, Node.js client receives internal Docker IP addresses on first contact which cannot be accessed from outside Docker on MacOs. See this previous blog post for more details.

If you're a Node.js user and want to store data remotely in Infinispan server instances, please give the client a go and tell us what you think of it via our forum, via our issue tracker or via Zulip on Infinispan channel.

Cheers,
Galder



Friday, 3 August 2018

Infinispan 9.4.0.Beta1 is out!

Infinispan users,

We have just released 9.4.0.Beta1 which includes bug fixes and improvements. Highlights of this release include:

  • Removal of WebSocket server support (ISPN-9386);
  • One step closer to remove compatibility mode, by dropping it from Remote Queries, Tasks and Scripts (ISPN-9180, ISPN-9182)
  • Recovery Support for Hot Rod client transactions (ISPN-9261)
  • Fixed issue with Hot Rod client near cache for async operations (ISPN-9393)
  • Improvements in Ickle (ISPN-9378)
  • Additional Segmented Stores
  • RocksDB supports single database segmentation (ISPN-9375)
  • RemoteStore segmented for additional stream performance (ISPN-9376)
  • RocksDB now allows for properties to be provided to configure underlying database (ISPN-9371)
  • Component Upgrades:
    • Protostream upgraded to version 4.2.1.Final (ISPN-9399)
    • Hibernate ORM upgraded to version 5.3.4.Final (ISPN-9406)
  • Other bug fixes.

The full list of 9.4.0.Beta1 fixes are here.

You can find both releases on our download page. Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.

Enjoy,
The Infinispan Team

Tuesday, 24 July 2018

Infinispan Spark connector 0.8 released


The Infinispan Spark connector version 0.8 has been released and is available in Maven central and SparkPackages.

This is a maintenance only release to bring compatibility with Spark 2.3 and Infinispan 9.3.

For more information about the connector, please consult the documentation and also try the docker based sample.

For feedback and general help, please use the Infinispan chat.



Monday, 16 July 2018

Infinispan 9.3.1.Final and 9.4.0.Alpha1

We have 2 new releases to announce today:

9.3.1.Final includes some important bug fixes, and we recommend all users of 9.3.0.Final to upgrade:
  • Fix for CVE-2018-1131 that allows unchecked deserialization in the server from binary java , XML and JSON payloads
  • Fixed transcoding from JSON/XML to java objects with deployed entities (ISPN-9336)
  • Look up key in cache loader if the entry has expired but hasn't yet been removed from the data container (ISPN-9370)
  • Avoid circular references in exceptions, as they were causing stack overflows with logback 1.2.x (ISPN-9362)
See the full list of bug fixes here.


9.4.0.Alpha1 its the first iteration towards our next big release. Highlights include:
  • The Spring Cache provider now supports two configuration properties with which you can determine how long to wait for read and write operations respectively (ISPN-9301).
  • You can now obtain nanosecond-resolution statistics for average read/write/remove time (ISPN-9352).
  • Queries now throw an AvailabilityException if the cache is in degraded mode and partition mode isn’t ALLOW_READ_WRITES ([ISPN-9340)
  • Admin Console: You can now delete cache from Administration console (ISPN-7291).
  • Following up on the segmented data container in 9.3.0.Final, cache stores can now be segmented as well, allowing for better performance for bulk operations (ie. cache.size(), cache.entrySet().stream())
  • The server-side Hot Rod parser is now generated automatically (ISPN-8981
The full list of 9.4.0.Alpha1 fixes is here.


You can find both releases on our download page. Please report any issues in our issue tracker and join the conversation in our Zulip Chat to shape up our next release.
 

Monday, 2 July 2018

Hotrod clients C++ and C# 8.3.0.Alpha1 are out!

Dear Infinispanners,

The C++ and C# 8.3.0.Alpha1 releases are available!

Both the clients come with these new features:
  • counter operations, to use cluster distributed counters [1]
  • admin operations, to create/remove cache programmatically at runtime [2]
For the .NET Core lovers, there's a work in progress to implement the dotnet core build for the C# client [3].
Features list, code and bits are available as usual: [4] [5] [6].

Cheers,
The Infinispan Team

[1] Clustered Counters
[2] Hot Rod Admin Tasks
[3] How to build à la .NET Core manière
[4] Release notes
[5++] C++ code for 8.3.0.Alpha1
[5#] C# code for 8.3.0.Alpha1
[6] Downloads

Wednesday, 27 June 2018

Making Java objects queryable by Infinispan remote clients

The following is a common question amongst Infinispan community users:
How do I make my Java objects queryable by remote clients? 

Annotation Method


The simplest way is to take advantage Infinispan Protostream annotations to mark your objects queryable and decide how each object field should be indexed. Example:

Then, the ProtoSchemaBuilder can inspect the annotated class and derive a Google Protocol Buffers schema file from it. Example:

Finally, the schema file needs to be registered in the “___protobuf_metadata” cache:

Although this is by far the easiest way to make your Java objects queryable, this method might not always be viable. For example, you might not be able to modify the Java object classes to add the annotations. For such use cases, a more verbose method is available that does not require modifying the source code of the Java object.

Plain Object Method


For example, given this Java object:

A Protocol Buffers schema must be defined where comments are used to define the object as queryable and decide how each field is indexed:

This method also requires a Protostream message marshaller to be defined which specifies how each field is serialized/deserialized:

This method still requires the Protocol Buffers schema to be registered remotely, but on top of that, the schema and marshaller need to be registered in the client:

Clearly, this second method is a lot more verbose and more laborious when refactoring. If any changes are made to the Java object, the marshaller and Protocol Buffer schema need to also be changed accordingly. This is done automatically in the first method.

Both methods are demonstrated in full in the queryable-pojos demo.

Cheers
Galder

Tuesday, 26 June 2018

Infinispan 9.3.0.Final is out!

We're delighted to announce the release of Infinispan 9.3.0.Final, which is a culmination of several months of hard work by the entire Infinispan community. Here's a summary of what you can find within it:

  • First final release to work with both Java 8 and Java 10. Note that Infinispan only works in classpath mode.
  • Transaction support Hot Rod. The java Hot Rod client can participate in Java transactions via Synchronization or XA enlistment. Note that recovery isn't supported yet.
  • Caches can now configure the maximum number of attempts to start a CacheWriter/CacheLoader on startup before cache creation fails.
  • Write-behind stores are now fault-tolerant by default.
  • Segmented On Heap Data Container. It improves performance of stream operations.
  • Server upgraded to Wildfly 13.
  • We have introduced several WildFly feature packs to make it easier for Infinispan to be utilised on WildFly instances via the Server Provisioning Plugin. The following feature packs have been created, most notably:
    • infinispan-feature-pack-client
      • All of the modules required to connect to a hotrod server via the client
    • infinispan-feature-pack-embedded
      • The modules required for embedded instances of Infinispan
    • infinispan-feature-pack-embedded-query
      • The same as above but with query capabilities
    • infinispan-feature-pack-wf-modules
      • This is equivalent to the Wildfly-modules.zip
  • Hibernate second-level cache provider works with Hibernate ORM 5.3.
  • The Hot Rod Server allows now to use multiple protocols with a Single Port. The initial version supports HTTP/1.1, HTTP/2 and Hot Rod. Switching protocols can be done using TLS/ALPN and HTTP/1.1 Upgrade header.
  • Admin console - improved all editors (schema, scripts, JSON data) to include syntax highlighting.
  • Several enhancements in the Java Hot Rod client allowing to read and write data in different formats such as JSON, for cache operations and deployed filters/converters.
  • Cluster wide max idle expiration.
  • Component Upgrades
    • Hibernate Search 5.10
    • Hibernate ORM 5.3
  • Numerous bug fixes which improve stability
For more details, please check our issue tracking release notes.

Thanks to everyone involved in this release! Onward to Infinispan 9.4!

Cheers,
Galder

Tuesday, 5 June 2018

Thanks Great Indian Developer Summit & Voxxed Days Zurich

A few days after Devoxx France, I headed for Great Indian Developer Summit in Bangalore where I spoke about handling streaming data on top of a Kubernetes platform. This was a very similar talk to the one I gave at JFokus but with some important changes. Together with Clement we created a small RxJava 2 façade for Infinispan. When combined with Vert.x RxJava 2 API, we could finally have an idiomatic way of handling streaming data asynchronously and coordinating events purely using RxJava 2 APIs. This is crucial for working with streaming data in an efficient way. On top of that, I made some changes to push the binary data used by the demo outside of the deployment.

Unfortunately Murphy struck during the presentation and I was unable to run the live coding demo. A problem with Docker size image during preparation combined with a cleanup I ran before the talk meant some of the images had to be re-downloaded. The wireless internet connection at the conference nor the mobile connection were good enough for me to recover it. Once back in the hotel where I had a stable connection I was able to record a screencast of the steps I would have followed during the Great Indian Developer Summit talk. You can find this screencast below:



The code from the demo can be found here. The live coding steps I followed are defined here. Finally the slides can be found here:

Finally, for something slightly different, back in March I joined Ray Tsang for a talk at Voxxed Days Zurich. This was a really fun talk to be part of! We combined past stories of my time at JBoss support with Ray's Kubernetes troubleshooting experience to create an engaging talk :). You can find the video below:



The week after Sebastian Łaskawiec and I travelled to Red Hat Summit as part of the work we did to integrate Red Hat Data Grid (Infinispan product version) into the Scavenger Hunt game presented on the keynote of the last day. Both Sebastian and I have been working on a blog series which will be published very soon.

Cheers,
Galder

Monday, 4 June 2018

Infinispan 9.3.0.CR1

Dear Infinispan Community,

we're glad to announce that 9.3.0.CR1 is out!

This is the first release which works with both Java 8 and Java 10. Pre-releases of Java 11 work too. Note that Infinispan still only works in classpath mode.

Highlights of this release include:
  • Expanded transaction support in Hot Rod, which can now participate in Java transactions via Sync or Xa enlistment. Transaction recovery isn't supported yet.
  • Caches can now configure the maximum number of attempts to start a CacheWriter/CacheLoader on startup before cache creation fails.
  • Write-behind stores are now fault-tolerant by default
  • Segmented On-Heap Data Container improves stream operation performance
  • We have introduced several WildFly feature packs to make it easier for Infinispan to be utilised on WildFly instances via the Server Provisioning Plugin. The following feature packs have been created:
    • infinispan-feature-pack-client: all of the modules required to connect to a hotrod server via the client
    • infinispan-feature-pack-embedded: the modules required for embedded instances of Infinispan
    • infinispan-feature-pack-embedded-query: the same as above but with query capabilities
    • infinispan-feature-pack-wf-modules: this is equivalent to the wildfly-modules.zip
  • 2-Level cache now works with Hibernate ORM 5.3
  • The server now allows multiple protocols with a Single Port. The initial version supports HTTP/1.1, HTTP/2 and Hot Rod. Switching protocols can be done using TLS/ALPN and HTTP/1.1 Upgrade header.
  • Admin console - improved all editors (schema, scripts, JSON data) to include syntax highlighting
  • Component Upgrades: Hibernate Search 5.10 and Hibernate ORM 5.3
Numerous bug fixes which improve stability are also included (here is the full list of the solved issues).

As usual, you can find all the bits on our website. If you find any issues, don't hesitate to report them on our issue tracker.

Friday, 11 May 2018

Infinispan 9.3.0.Beta1

Infinispan users,

We have just released 9.3.0.Beta1 which includes 38 fixes. Highlights of this release include:
  • Conflict Resolution Improvements
    • MergePolicy.NONE is now the default merge-policy
    • Conflict Resolution during a merge is now non-blocking and tolerant of node failures
  • Reactive Streams based Cache Loader SPI available
  • Infinispan can now be built and tested with Java 10/11
  • Max Idle expiration is now cluster-wide including events
  • The Java Hot Rod client can handle data in multiple formats
  • Improved merge after long GC pauses avoiding data loss
  • Admin console supports counters in standalone mode
  • Lots of bug fixes, test fixes, and documentation improvements
As usual, you can find all the bits on our website. If you find any issues, don't hesitate to report them on our issue tracker.

Enjoy,
The Infinispan Team

Wednesday, 2 May 2018

Infinispan chat moves to Zulip

For over 9 years Infinispan has used IRC for real-time interaction between the development team, contributors and users. While IRC has served us well over the years, we decided that the time has come to start using something better. After trying out a few "candidates" we have settled on Zulip



Zulip gives us many improvements over IRC and over many of the other alternatives out there. In particular:
  • multiple conversation streams
  • further filtered with the use of topics
  • organization management to organize users into groups
  • it's open source

So, if you want to chat with us, join us on the Infinispan Zulip Organization.

Infinispan 9.2.2.Final and 9.3.0.Alpha1 are out

We have two releases to announce:

first of all is 9.2.2.Final which introduces a second-level cache provider for the upcoming Hibernate ORM 5.3 as well as numerous bugfixes. [1]

Next is 9.3.0.Alpha1 which is the first iteration of our next release.
The main item here, aside from bugfixes and preparation work for upcoming features, is the upgrade of our server component to WildFly 12.

Go and get them on our download page [3]

Monday, 30 April 2018

Danke Javaland, Merci Devoxx France!

Ever since we started the Infinispan project, I simply don't recall a time busier than the current one! As usual we're working on new features and supporting our community and enterprise users, but on top of that we're working on a big scale demo that we hope you are going to like it!

All of that has been happening with at the same time juggling one of the busiest periods in Infinispan evangelisation that I can remember! Here's what I have been up to past month and a half:


On 18th of March I gave a brand new presentation at Javaland where I explored the Java RPC framework landscape. Since 2010, Infinispan has offered a binary remote, client/server API for interacting with the data. As more clients have been developed, we've been noticing that we're spending more time that'd we'd like implementing features across different clients. In this talk I looked at existing Java RPC frameworks from the point of view of Infinispan's remote API requirements. If we were to implement our binary protocol again, which option would fit best? The spectrum can be very vast so I limited myself to some know options (HTTP 1.1 REST), some options we have expertise on (Netty), some upcoming players (gRPC) and some lesser known but very powerful players (Aeron). The talk was not recorded but the slides can be found here:



Javaland was a very enjoyable conference set in a theme park near Cologne. The sessions were a mix in German and English. Although my German is not very good, I was able to follow some of the sessions.
After Javaland, I turned my attention to Devoxx France which was held in Paris mid-April. I had a couple of sessions at Devoxx France. The first was a streaming data analysis 3 hour university talk delivered along with Clement Escoffier from the Vert.x team. We already delivered this session at JFokus earlier this year, but at Devoxx France it was recorded so you'll soon to be able to watch it! In the mean time, slides can be found below:

On top of that, Ray Tsang from Google joined Clement and I to deliver a 3 hour streaming data hands-on lab at Devoxx France. This was the same workshop we delivered in Devoxx Belgium and Codemotion Madrid in 2017. The big difference was that instead of having the users run it locally on their laptops within a virtual machine, they could run OpenShift and all the components inside of Google Cloud Platform. The users had a better experience as a result of not having to deal with a virtual machine :). I'd like to say special thanks to Ray for the Google Cloud Platform test accounts and the support during the workshop.


Finally, many thanks to all attendees that came to the sessions, and to the organisers/sponsors for creating two outstanding events!

In the next blog post I'll be talking about Great Indian Developer Summit and Voxxed Days Zurich. Stay tuned! :)

Cheers,
Galder

Thursday, 29 March 2018

Infinispan Spark connector 0.7 released!

A new version of connector that integrates Infinispan and Apache Spark has just been released!

This release brings compatibility with Infinispan 9.2.x and Spark 2.3.0.

Also included a new feature that allows to create and delete caches on demand, passing custom configurations when required. For more details, please consult the documentation.

To quickly try the connector, make sure to check the Twitter demo, and for any issues or suggestions, please report them on our JIRA.

Cheers!


Tuesday, 27 March 2018

Infinispan 9.2.1.Final

Infinispan users,

we have just released 9.2.1.Final which includes 65 fixes. Highlights of this release include:
  • Many fixes/improvements to the REST endpoint
    • Configurable CORS settings
    • HTTP/2 now works
    • Accept-Encoding and Content-Encoding handling
  • It is now possible to retrieve the list of cache names over Hot Rod
  • Substantial performance improvements when iterating over the file store
  • Lots of bug fixes, test fixes and documentation improvements
As usual you can find all the bits on our website. If you find any issues, don't hesitate to report them on our issue tracker.

Enjoy

The Infinispan Team

Wednesday, 21 March 2018

Clustering Vert.x with Infinispan

Welcome to the third in a multi-part series of blog posts about creating Eclipse Vert.x applications with Infinispan. In the previous blog posts we have seen how to create REST and PUSH APIs using the Infinispan Server. The purpose of this tutorial is to showcase how to create clustered Vert.x applications using Infinispan in embedded mode.


Why Infinispan ?


Infinispan can be used for several use cases. Among them we find that it can be used as the underlying framework to cluster your applications. Infinispan uses peer-to-peer communication between nodes, so the architecture is not based on master/slave mode and there is no single point of failure. Infinispan supports replication and resilience across data centers, is fast and reliable. All the features that make this datagrid a great product, make it a great cluster manager. If you need to create clustered applications or microservices, this can be achieved with Vert.x using the Vert.x-Infinispan cluster manager.


Creating a clustered application


The code of this tutorial is available here.

Dummy Application


Let’s start with a dummy clustered system with 3 verticles.


WebService Status Producer

Produces randomly [0,1,2] values every 1000 milliseconds and sends them to the event bus "ids" address.



Reboot Consumer

Consumes messages from the event bus "ids" address, and launches a "reboot" that lasts for 3000 milliseconds whenever the value is 0. If a reboot is already happening, we don’t need to relaunch any new reboot. When a reboot starts or ends, a message is sent to the event bus to the "reboot" address.

Notice that:
  • We use a simple boolean to check if there is a reboot going on. This is safe because every verticle is executed from a single event loop thread, so there won’t be multiple threads executing the code at the same time.
  • An ID is generated to identify the Verticle. The message sent to the event bus is a JsonObject



Monitoring

Consumes monitoring messages from the event bus "reboot" address and logs them.



Clustering the dummy Application


To create a cluster of these applications, we just need to do 2 things:

  1. Add the cluster manager maven dependency.  
  2. Run and deploy each verticle in cluster mode. Each Verticle class has a main method that deploys each verticle separetly. Example for the Monitoring verticle.

Running the application, we can monitor the logs
Each clustered application contains - or embeds - an Infinispan instance. Under the hood, the 3 Infinispan instances will form a cluster.


What if I need to scale


Imagine you need to scale the Reboot Consumer application. We can run it multiple times, let’s say 2 more times. The two new instances will join the cluster. In this case, we have “ID-93EB” ”ID-45B8” and “ID-247A”, so now we have a cluster of 5. It's very simple but if we have a look to the monitoring console, we will notice reboots are now happening in parallel.

3 Reboot Consumers


As I mentioned before, this example is a dummy application. But in real life you could need to trigger a process from a verticle that runs multiple times and need to be sure this process is happening just once at a time. How can we fix this ? We can use Vert.x Shared Data structures API.

Shared data API to rescue


In this particular case, we are going to use a clustered lock. Using the lock, we can now synchronise the reboots among the 3 nodes.


Using Shared Data API, one reboot at a time

Vert.x clustered lock in this example is using an emulated version of the new Clustered Lock API of Infinispan introduced in 9.2 which has been freshly released. I will come back to share about this API in particular in further blog posts. You can read about it on the documentation or run the infinispan-simple-tutorial.

One node at a time


When clustering applications with Vert.x, there is something you need to take care of. It is important to understand that each node contains an instance of the datagrid. This means that scaling up and down needs to be done one at a time. Infinispan, as other datagrids, reshuffles the data when a new node joins or leaves a cluster. This process is done following a distributed hashing algorithm, so not every data is moved around, just the data that is supposed to live in the new node, or the data owned by a leaving node. If we just kill a bunch of nodes without taking care of the cluster, consequences can be harming! This is something quite obvious when dealing with databases : we just don’t kill a bunch of database instances without taking care of every instance at a time. Even when Infinispan data is only in memory we need to take care about it in the same way. Openshift, which is built on top of Kubernetes, helps dealing properly and safely with these scale up and down operations.


Conclusions


As you have seen, creating clustered applications with Vert.x and Infinispan is very straightforward. The clustered event bus is very powerful. In this example we have seen how to use a clustered lock, but other shared data structures built on top of Counters are available.


About the Vert.x Infinispan Cluster Manager status


At the time of this writing, Infinspan 9.2.0.Final has been released. From vert.x-infinispan cluster manager point of view, before Vert.x 3.6 (which is not out yet) the cluster manager is using Infinispan 9.1.6.final and it’s using an emulation layer for locks and counters. In this tutorial we are using Vert.x 3.5.1 version.

This tutorial will be updated with the version using Infinispan 9.2 as soon as the next vert.x-infinispan will be released, which will happen in a few months. Meanwhile, stay tuned!