Big data analysis can be vexing even for the most technical of teams, and Google has acknowledged this with a BigQuery update to help simplify the process.
Big JOIN, Big Group Aggregations
For those with terabytes of data to analyze, BigQuery now has the ability to merge data sets from two large tables using a common key. It's called Big JOIN, and it helps do this without the need for batch based processing, Michael Manoochehri, developer programs engineer for Google's cloud platform wrote in the company's Developer Blog.
BigQuery already had the ability to join a large and a small data set together, so Google has now simply made it possible to do the same for multiple large sets.
The Big Group Aggregations tool helps with dividing user groups from voluminous user activity logs so they can be analyzed. Web applications sometimes produce millions of lines of user activity log code in just hours, and breaking it up for analysis can be cumbersome. Not anymore.
This could be particularly helpful for an e-commerce site, for example. With potentially tens of thousands of user buying habits, the Group function helps to combine them in new and interesting ways for gaining usable insights.
A popular request from developers has also been met, Manoochehri wrote, and a native TIMESTAMP data type has been introduced to allow for importing of date and time values that keep timezone offset info.
Direct links have been added to the BigQuery Web UI so teams can share datasets, and it includes bookmarking and sharing. Additionally, email notifications have been added to tell people when they've been granted access to a dataset.
Granular controls for sharing datasets introduced in BigQuery update.
Data analysis with the BigQuery API should be as simple as possible, and that is in keeping with Google's philosophy overall on many things. One of the things that still needs work, however, is the BigQuery public documentation. The team is working on this however, and there are several tools soon to be added to make it more wieldy.
Below is a 16 minute video with more details on the updates, and there are some demos for each one as well.