By admin 
In an era where data flows at an unprecedented scale, AT&T, a titan of the telecommunications industry, has confronted a monumental challenge: managing an average daily token usage of 8 billion. This staggering volume presented a critical problem for Chief Data Officer Andy Markus and his team. They recognized that attempting to process such immense amounts of data through monolithic, large reasoning models was not only economically prohibitive but fundamentally infeasible. This realization spurred a strategic re-architecture of their internal systems, culminating in the development of "Ask AT&T," a sophisticated personal assistant powered by a novel multi-agent stack built on the LangChain framework.
At the core of this innovation lies a meticulously designed orchestration layer. Instead of relying on a single, powerful large language model (LLM) to handle every task, AT&T has implemented a hierarchical structure. "Super agents," which are themselves sophisticated LLMs, act as intelligent dispatchers. They analyze incoming requests and delegate specific, well-defined sub-tasks to a legion of smaller, more specialized "worker" agents. These worker agents are purpose-built to perform concise, focused operations, such as data retrieval, specific calculations, or targeted analysis. This modular approach, where complexity is broken down into manageable components, has yielded remarkable improvements in performance. Markus reported to VentureBeat that this flexible orchestration layer has dramatically reduced latency, enhanced speed, and significantly improved response times. The most striking outcome, however, has been a staggering cost saving of up to 90%, a testament to the economic advantages of this distributed AI architecture.
Markus is a firm believer in the transformative potential of smaller, specialized AI models, stating, "I believe the future of agentic AI is many, many, many small language models (SLMs)." He further elaborated on the effectiveness of this strategy: "We find small language models to be just about as accurate, if not as accurate, as a large language model on a given domain area." This assertion underscores a key principle of AT&T’s AI strategy: precision over brute force. By leveraging the focused capabilities of SLMs for specific tasks, they achieve comparable accuracy to larger models but with a fraction of the computational resources and cost.
Building on this foundation, AT&T has recently introduced "Ask AT&T Workflows." This new offering, developed in collaboration with Microsoft Azure, provides employees with a graphical, drag-and-drop interface for building and deploying automated workflows. This empowers a broad spectrum of employees, regardless of their coding expertise, to leverage AI for task automation. The underlying agents within Ask AT&T Workflows draw upon a rich suite of proprietary AT&T tools. These tools are designed to handle critical business functions such as advanced document processing, seamless natural language-to-SQL conversion for data querying, and sophisticated image analysis.
The integration of these proprietary tools with the agentic architecture ensures that decisions are not made in a vacuum but are deeply rooted in AT&T’s own operational data. "As the workflow is executed, it’s AT&T’s data that’s really driving the decisions," Markus emphasized. This approach shifts the paradigm from asking general questions to posing specific inquiries directed at the company’s vast data repositories. By bringing their data to bear, the AI is guided to focus on relevant information, ensuring that decisions are contextually accurate and strategically aligned with business objectives.
Despite the increasing autonomy of these AI agents, a crucial human oversight remains integral to the process. Markus described this as a "chain reaction" where a human remains in the loop to monitor and validate the entire sequence of agent actions. Every action taken by an agent is meticulously logged, ensuring transparency and auditability. Data is rigorously isolated throughout the workflow execution, maintaining data integrity and security. Furthermore, role-based access controls are strictly enforced, governing how agents interact and transfer workloads amongst themselves, preventing unauthorized access or actions. "Things do happen autonomously, but the human on the loop still provides a check and balance of the entire process," Markus reassured, highlighting the commitment to responsible AI deployment.
A fundamental aspect of AT&T’s AI philosophy is a pragmatic approach to development, avoiding the temptation to "build everything from scratch." Instead, the company prioritizes the utilization of models that are "interchangeable and selectable," and crucially, "never rebuilding a commodity." This agile methodology allows AT&T to adapt rapidly to the ever-evolving AI landscape. As new functionalities mature and become readily available as off-the-shelf solutions, they are prepared to deprecate homegrown tools in favor of these more advanced, industry-standard options.
"Because in this space, things change every week, if we’re lucky, sometimes multiple times a week," Markus explained, underscoring the dynamic nature of AI development. "We need to be able to pilot, plug in and plug out different components." This adaptability is a strategic imperative, enabling them to quickly integrate cutting-edge AI capabilities and remain at the forefront of technological advancement.
AT&T conducts "really rigorous" evaluations of both external and internal AI solutions. Their commitment to excellence is evidenced by achievements such as their "Ask Data with Relational Knowledge Graph" topping the Spider 2.0 text-to-SQL accuracy leaderboard. Similarly, other in-house tools have demonstrated strong performance on benchmarks like the BERT SQL benchmark, validating the effectiveness of their development and evaluation processes.
For their homegrown agentic tools, LangChain serves as the foundational framework. The team employs standard retrieval-augmented generation (RAG) techniques and other proprietary algorithms for fine-tuning models. Their partnership with Microsoft is also pivotal, leveraging the tech giant’s search functionality for their vector store, a critical component for efficient data retrieval and knowledge management.
Markus also offered a cautionary note against the indiscriminate application of advanced AI technologies. He advised against integrating agentic AI or other sophisticated tools simply for the sake of adoption. "Sometimes we over complicate things," he admitted. "Sometimes I’ve seen a solution over engineered." He urged builders to critically assess whether a given task truly necessitates an agentic solution. This involves posing fundamental questions: What level of accuracy could be achieved with a simpler, single-turn generative solution? Can the task be broken down into smaller, more manageable components, each of which can be delivered "way more accurately"?
The core principles guiding their development remain focused on accuracy, cost-effectiveness, and tool responsiveness. "Even as the solutions have gotten more complicated, those three pretty basic principles still give us a lot of direction," Markus stated, reinforcing the enduring importance of foundational engineering disciplines in the pursuit of advanced AI.
The impact of Ask AT&T Workflows is already being felt across the organization, with over 100,000 employees now having access to the platform. A significant portion, more than half, report using it daily, and active adopters are witnessing productivity gains as high as 90%. Markus tracks user engagement through metrics like "stickiness," which measures the repeated use of the system, identifying it as a strong indicator of success and value.
The agent builder offers a dual approach to user engagement, catering to different technical proficiencies. The "pro-code" journey allows users to delve into Python programming, enabling them to define intricate rules and logic for agent behavior. Complementing this is the "no-code" option, which features an intuitive drag-and-drop visual interface designed for a "pretty light user experience," according to Markus.
An intriguing observation has emerged from user behavior: even highly proficient technical users are increasingly gravitating towards the no-code interface. During a recent hackathon targeting a technical audience, participants were given the choice between both options. To the surprise of the AT&T team, more than half opted for the low-code approach, despite their strong programming capabilities. This preference suggests that the ease of use and rapid prototyping offered by the no-code interface hold significant appeal, even for experienced developers.
The application of these AI agents spans a diverse range of AT&T’s operational functions. For instance, a network engineer might deploy a series of agents to autonomously address network alerts and restore customer connectivity. In such a scenario, one agent could correlate telemetry data to pinpoint the network issue and its exact location, then retrieve change logs and check for known issues. Following this analysis, it could automatically open a trouble ticket. A subsequent agent might then devise potential solutions, even generating new code to patch the identified vulnerability. Once the problem is resolved, a final agent could compile a comprehensive summary report, including preventative measures to mitigate future occurrences. In all these instances, the human engineer remains in control, observing the agents’ performance and intervening as necessary to ensure optimal outcomes.
This engineering discipline of deconstructing complex tasks into smaller, purpose-built components is now fundamentally reshaping how AT&T approaches software development itself, a process Markus dubs "AI-fueled coding." He draws a parallel between this methodology and RAG, where developers utilize agile coding practices within an integrated development environment (IDE). This is coupled with "function-specific" build archetypes that meticulously define how code components should interact.
The output of this AI-fueled coding process is not rudimentary or fragmented code. Instead, the generated code is described as "very close to production grade," often achieving this quality in a single iteration. Markus contrasts this with "vibe coding," where code editors offer agentic assistance, but often result in significant back-and-forth iterations. AI-fueled coding, he argues, "eliminates a lot of the back and forth iterations that you might see in vibe coding."
This novel coding technique is seen as "tangibly redefining" the entire software development lifecycle. It promises to significantly shorten development timelines and boost the output of production-ready code. Moreover, it democratizes software creation, enabling non-technical teams to participate by using plain language prompts to build software prototypes. As a concrete example, Markus cited his team’s experience in building an internal curated data product in a mere 20 minutes, a task that would have traditionally required six weeks of development time. "We develop software with it, modify software with it, do data science with it, do data analytics with it, do data engineering with it," Markus concluded, emphasizing its broad applicability and transformative impact. "So it’s a game changer."