Google Service – Live streaming data analysis
There are many options for companies and sites to analyze the data that gets uploaded, however seems like Google is one step ahead and planning for Google cloud Dataflow service that will enable to not only analyze live streaming data and batch data. This can certainly help the users of this service to change based on the current trends and make their moves.
According to Brian Goldfarb, head of the Google cloud Dataflow service marketing, as much as different data gets created, it becomes really important to ingest and secure the appropriate and important ones. When it comes to analyzing large data, it involves usage of different program models and technologies. But at the end the managers of this service will be able to learn and implement a lot new services.
Google cloud Dataflow:
This is completely managed service which enables one to create data pipelines in order to store and ingest the data and either live-streaming mode or batch mode. This service is meant for analyzing random amount of data’s. This service will enable to user to focus more on the analysis rather than giving importance to pipeline maintenance and processing infrastructure. This service can be used for measuring unusual activity in the form of security tool or by companies to analyze the emotions of consumers towards any of their products on a particular social networking platform. This service can be included in many other business applications and can well be used as an alternative service to ETL.
Advantages:
This service is based on the Mapreduce programming model which is currently being used in Apache Hadoop and the technologies which was developed by Google to use them internally. Through Hadoop large amount of data through different servers can be analyzed and pioneered the area of analyzing data even though it initially used to focus on writing the data and that too in batch mode. The limitation is reached when all the data needs to be collected before it could be analyzed.
Google has been developing and taking a different approach when it comes to live streaming data analysis, through incorporating different technologies including Flume and MillWheel which has been built by the company itself. Flume has been developed to store large amount of data and on the other hand Millwheel helps in providing a platform for data analysis.
How these will work:
This service will provide a platform (software development kit) which can be used to develop complex pipelines and perform analysis. This service will also be based on Java programming language. Currently it doesn’t support any other language. This will work as a library which will enable the users to store large amount of data’s from different sources and later on analyze it. This can be queried against Google’s own Bigquery service. User of this service can analyze the current trends by writing modules to examine the stored data.
Even though currently this service is only used by certain selected Google users, it might be available on public platform later on.
