• Cloud Database Insider
  • Posts
  • 70% Overspend in Enterprise Databricks Use Revealed|AI Boom Pushes Firms to Tackle Data Chaos Now|Redis Unveils IRIS AI Engine

70% Overspend in Enterprise Databricks Use Revealed|AI Boom Pushes Firms to Tackle Data Chaos Now|Redis Unveils IRIS AI Engine

Deep Dive: The Agentic AI situation and the Data Practitioner

In partnership with

What’s in today’s newsletter:

70% overspend on Databricks; optimize costs now 💸

AI boom drives firms to fix data chaos urgently ✨

Redis launches IRIS to boost AI context understanding 🤖

Also, check out the weekly Deep Dive - Agentic AI and the realm of the Data Professional

Give Your AI Agent Eyes on the Web

MCP servers eat 72% of your agent's context window before it reads a single user message. There's a simpler way.

Bright Data CLI gives coding agents like Claude Code, Cursor, and Copilot direct access to real-time web data - from the terminal. No MCP schema bloat. No server setup. Just one command:

Scrape any URL with automatic CAPTCHA bypass. Search Google/Bing/Yandex. Extract structured data from 40+ platforms (Amazon, LinkedIn, Instagram, TikTok, YouTube, Reddit, and more).

One install. Works with 46+ AI agents. 10-32x cheaper than MCP for the same tasks.

DATABRICKS

TL;DR: Lucent Innovation finds 70% of enterprise Databricks deployments overspend due to inefficient resource use and weak cost governance, urging cost monitoring and optimization to sustain ROI and manage cloud analytics budgets effectively.

  • 70% of enterprise Databricks deployments exceed budgets due to inefficient resource use and poor cost governance.

  • Overspending arises from misaligned business needs and wasted compute power or storage resources.

  • Lucent Innovation recommends cost monitoring and optimization strategies for more effective Databricks management.

  • Addressing overspend is crucial for sustaining ROI and supporting broader IT and analytics initiatives.

Why this matters: Overspending on Databricks threatens enterprise budgets and limits data analytics progress. Without cost governance and optimization, companies waste resources, risking ROI and stalling projects. Lucent Innovation’s insights emphasize the urgent need for strategic cost control to ensure sustainable, efficient cloud analytics investments.

DATA LINEAGE

TL;DR: The AI boom exposes widespread data chaos, pushing firms to improve data governance, standardization, and IT-business collaboration to unlock AI’s full potential and gain strategic advantages.

  • The AI boom exposes data chaos in firms, highlighting unstructured and poorly managed data issues.

  • Success with AI requires resolving data silos, standardizing formats, and improving data governance.

  • Addressing data chaos enhances decision-making, operational efficiency, and unlocks value from data assets.

  • IT-business collaboration increases, enabling agile responses and better leveraging AI technologies.

Why this matters: The AI boom reveals critical data management gaps that firms must fix to succeed with AI. By resolving data chaos and fostering IT-business collaboration, companies improve decision-making and efficiency, unlocking greater value and gaining a strategic edge in an increasingly AI-driven competitive landscape.

IN-MEMORY DATABASE

TL;DR: Redis Labs launched the IRIS context engine to enhance AI agent workloads with real-time, scalable context processing, improving response relevance and productivity across industries using large language models.

  • Redis Labs launched the IRIS context engine to enhance AI agent workloads with real-time data and contextual understanding.

  • IRIS integrates real-time processing, advanced search, and vector similarity search to enrich AI model responses.

  • The engine supports interoperability and scalability, catering to industries like customer support and personalized recommendations.

  • IRIS aims to improve AI response relevance and productivity, solidifying Redis’s role in the evolving AI ecosystem.

Why this matters: Redis’s IRIS context engine addresses critical data challenges in AI, enhancing real-time, context-aware responses that boost productivity and user experience. By integrating with existing systems and supporting scalability, it positions Redis as a key infrastructure provider for advanced AI agents, accelerating AI adoption across industries.

EVERYTHING ELSE IN CLOUD DATABASES

DEEP DIVE

My take on Agentic AI and the realm of Enterprise Databases

I’m not too worried. That is pretty much my take in a nutshell…

But seriously, I am not surprised by the advent of various types of databases being used as a back plane of sorts, to be part of the architecture of agentic AI workloads.

I have mentioned before, the blurring of the lines of cloud platforms that have tightly integrated database pieces and AI and ML components working closely together.

My only bone to pick is there is an incessant discussion in MSM that agentic AI is going to take away jobs, destroy the environment with the proliferation of data centers, etc. etc.

My suggestion is to have a measured embrace of Agentic AI, and figure out a way to incorporate your skills and knowledge as a data practitioner into the the agentic ecosystem.

There are are so many types of data and database platforms that there will always be a need for the data professional.

Think of it this way too; there is no set standard querying language as of yet for agentic AI applications to utilize and store data.

That still leaves even more of an opportunity for us database people to thrive and survive.

Again, I am not too worried about Agentic AI taking my job. Just remember that all the data artificial intelligence and agents use, has to be stored somewhere.

To all my American subscribers, I hope you have a peaceful and meaningful Memorial Day.

Gladstone Benjamin

🚀 Work With Cloud Database Insider

Looking to reach enterprise data engineers and architects?

Limited sponsorship slots available each month.