Since the last one year or so I have been involved in defining the platform vision, features and go to market strategy for HSDP – Hitachi’s Streaming Data Analytics Platform. Streaming analytics is not something new, however, the range of applications and use cases they can be used has increased in scope. As noted by Forrester – The market for streaming analytics platforms is growing far beyond its roots in industrial operations and financial services.
Since 2009 the HSDP was used to realize streaming solutions for a variety of verticals which included Finance, Energy, Utilities, Banking, IT ops, Transportation, Smart Cities. Many of the solution that were built were IoT type solutions.
Based on our experience with building customized streaming solutions for our customers the following are some of the requirements that we repeatedly observed
- Ability to ingest from a variety of data sources and publish insights to a variety of targets
- Ability to perform multi-stage geo distributed computations and the ability to provide and cascade insights from edges to center
- Ability to process (for example, filtering, enriching), perform descriptive analysis (for example, aggregations, statistical computations) and predictive analysis (for example, classifications, clustering) at the edges and center and anywhere inbetween.
- Ability to securely handle large volumes and velocity of data with minimal or no disruptions
- Ability to be deployed on hardware with limited resources and on commodity hardware in a clustered environment deployed on-premises or in the cloud
- Flexibility for integrating with a variety of technologies which includes OSS, visualization systems, decision systems, control systems, data stores etc.
- Support for solution development, deployment, management and monitoring
Realizing the the current market trend and based on our experience in building custom streaming solution, the following were some of the key goals that I set for the streaming platform
- Transform the platform into an enterprise class platform that addresses the requirements we observed.
- Simplify the user experience when using the platform for building solutions
- Keep the platform flexible so it can blended with the choices the solution provider or integrator makes when building and deploying specific IoT solution.
With the above goals in mind, I collaborated with the engineering architects @ Hitachi and defined the features and the user experience for the platform.
We are now on track for releasing the product on May 3rd, 2016.An introduction to the platform and its capabilities can be found here.