To effectively use the massively parallel processing (MPP) architecture with a SQL pool, you need to understand table geometries. Table geometries specify how data is sharded into distributions on your existing compute nodes. These table…
For customers who have an existing solution based on Apache Kafka for HDInsight and want to move it to Azure, this is the product of choice. From a Microsoft perspective, the same solution can be…
FIGURE 4.1 Azure Synapse Analytics compression Table 4.1 shows the Azure Synapse Analytics file formats and their supported codecs. TABLE 4.1 Supported codecs by file format Format bzip2 (.bz2) gzip (.gz) Deflate(.deflate) ZipDeflate(.zip) TarGZip(.tar/.tar.gz) TAR(.tar)…
FIGUER 3.78 Provisioning an Azure Stream Analytics job Consider testing the connection from the Azure Stream Analytics job and the event hub by pressing the Test menu option on the Input blade. The configuration of…
Streaming, batching, and querying data are all very performant and reliable activities when using delta tables. This means if your data analytics solution requires streaming or batching, the approach for ingesting, transforming, and serving is…
It is possible to create an additional policy with fewer rights. The RootManageSharedAccessKey policy has full access. You might consider creating a policy with the minimum access necessary, perhaps Send Only. It depends on your…
A sliding window is illustrated in Figure 3.82 and implemented using the following SQL snippet: SELECT READINGTYPE, COUNT(*) as CountFROM brainwaves TIMESTAMP BY CreatedAtGROUP BY READINGTYPE, SlidingWindow(second, 10)HAVING COUNT(*)> 3 The second parameter represents the…
Figure 3.75 shows how data is ingested into Azure Databricks. There are numerous ways to ingest data, including by streaming via Event Hubs, Stream Analytics, or Kafka, or by copying data from a remote source…
The Users page provides a list of users who have access to the workspace. The role and permissions of each user are rendered as well. Table 3.18 describes the permissions (i.e., entitlements). TABLE 3.18 Azure…
The Workspace section provides access to the assets that exist within the workspace. For example, you created an Azure Databricks workspace in Exercise 3.14. Depending on your permissions, you can find shared workspaces or workspaces…