Google recently announced the preview of a new feature called Log Analytics in its Cloud Logging service, allowing companies to analyze data collected from their cloud environments.
Cloud Logging is a fully-managed real-time log management service that enables storage, search, analysis, and alerting. With the Logs Explorer feature, for instance, users can search, sort, and query logs. The added Log Analytics powered by BigQuery allows them to perform advanced analytics using SQL to query logs. For example, they can run a query to determine the average latency of the requests sent to a cloud application. In addition, there is a new user interface optimized for analyzing log data (private preview).
When using BigQuery, users need to analyze their logs with data stored outside of Logging. This can be accomplished by upgrading a log bucket to use Log Analytics and creating a linked dataset. Subsequently, with the linked dataset, users can join their log data with other data, such as a dataset that stores known malicious URLs or data generated from business intelligence tools like Looker and Data Studio.
Mimoune Djouallah, a business intelligence analyst at Downer, tweeted:
Google Cloud introduced Log analytics; use #bigquery to analyze your data, notice the data is not duplicated as it is using a linked dataset, Querying the data in log analytics itself is free, again using Standard SQL!!! Google Cloud analytics is another league!!!
In addition, Charles Baer, a product manager at Google Cloud, explains in a Google blog post that Log Analytics is powerful for its:
- Centralized logging: there is no need for duplicate copies since log data is collected and centrally stored in a dedicated log bucket
- Reduced cost and complexity, as data can be reused across the organization
- Ad hoc analysis: users can perform ad-hoc query-time log analysis
- Scalable platform, as it can scale for observability using the serverless BiqQuery platform.
Google’s competitor in the public cloud space, Microsoft, has a similar service in Azure also called Log Analytics (part of Azure Monitor). It is a tool in the Azure portal allowing users to edit and run log queries against data in the Azure Monitor Logs store. The queries are performed with a proprietary Kusto query language (KQL).
The pricing of Log Analytics is included in the standard Cloud Logging pricing. According to the company, queries performed through the Log Analytics user interface do not incur additional costs. Furthermore, enabling analysis in BigQuery is optional. If enabled, queries submitted against the BigQuery linked data set, including Data Studio, Looker, and via BigQuery API, incur the standard BigQuery query cost.
Lastly, users can get started using available sample queries and sign up for charting capability in Log Analytics, which is still in private preview.
