Mainframes are different. They are the largest, fastest, and most capable systems on the planet, and they still run many, if not most, of the most important applications in the commercial world. One reason mainframes remain dominant? Their long history of efficient handling of business data. In large business data processing environments, it is a … Read more
Today we’re excited to welcome the Stately Cloud team to Databricks. This transaction continues to build on our deep investment in the core Databricks Data Intelligence Platform. Operating critical systems at scale The Stately Cloud team brings deep experience building and operating some of the largest distributed systems in the world, including Amazon and Snapchat. … Read more
Ryan and I have talked about a lot since we took over as editors of the Smart Data Collective a few years ago. One thing we haven’t talked about as much is why migrating to Microsoft 365 is important for organizations that rely on data to make decisions. You may have already noticed that many … Read more
Introduced in Spark 3.4, Apache Spark Connect enhances the Spark ecosystem by offering a client-server architecture that separates the Spark runtime from the client application. Spark Connect enables more flexible and efficient interactions with Spark clusters, especially in scenarios where direct access to cluster resources is limited or impractical. A key use case for Spark … Read more
Today’s consumers are faced with an incredible assortment of choices when shopping for everything from toothpaste to toys. Profitability depends on capturing shoppers in the moment and encouraging them to choose specific products or brands. A customer data platform (CDP) makes this possible by aggregating data from multiple online and offline sources to create profiles … Read more
Data collaboration is the backbone of modern AI innovation, especially when organizations work with external partners to uncover new insights. However, data privacy and intellectual property protection remain major challenges to enabling collaboration while protecting sensitive data. To bridge this gap, customers across industries are using Databricks Clean Rooms to perform shared analysis of sensitive … Read more
There is a popular saying that I personally don’t believe in – “Smart is slow”. Everything associated with high speed is somehow held in a negative light, just because it’s, well, fast. What they tend to forget – In today’s fast-paced world, speed may be your only ticket to success. This applies to humans, their … Read more
Deep learning models are based on activation functions that provide non-linearity and enable networks to learn complicated patterns. This article will cover the Softplus activation function, what it is and how it can be used in PyTorch. Softplus can be said to be a smooth form of the popular ReLU activation that mitigates the disadvantages … Read more
AWS recently announced the general availability of automatic optimization for the Amazon OpenSearch Service vector engine. This feature streamlines vector index optimization by automatically evaluating configuration tradeoffs between search quality, speed, and cost savings. You can then run the vector intake pipeline to create an optimized index for the desired collection or domain. Previously, optimizing … Read more
Since the last month, Gemini 3 announcements seem to be everywhere. Google is pushing its latest AI model (and its subsets like the Nano Banana Pro) through all possible marketing channels, which is one of the reasons why we can see its LED billboards, posters, online ads and even ads during screenings between India and … Read more