Connect with us

Funding & Investment in Travel

OpenCUA’s open source computer-use agents rival proprietary models from OpenAI and Anthropic

Published

on


Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now


A new framework from researchers at The University of Hong Kong (HKU) and collaborating institutions provides an open source foundation for creating robust AI agents that can operate computers. The framework, called OpenCUA, includes the tools, data, and recipes for scaling the development of computer-use agents (CUAs).

Models trained using this framework perform strongly on CUA benchmarks, outperforming existing open source models and competing closely with closed agents from leading AI labs like OpenAI and Anthropic.

The challenge of building computer-use agents

Computer-use agents are designed to autonomously complete tasks on a computer, from navigating websites to operating complex software. They can also help automate workflows in the enterprise. However, the most capable CUA systems are proprietary, with critical details about their training data, architectures, and development processes kept private.

“As the lack of transparency limits technical advancements and raises safety concerns, the research community needs truly open CUA frameworks to study their capabilities, limitations, and risks,” the researchers state in their paper.


AI Scaling Hits Its Limits

Power caps, rising token costs, and inference delays are reshaping enterprise AI. Join our exclusive salon to discover how top teams are:

  • Turning energy into a strategic advantage
  • Architecting efficient inference for real throughput gains
  • Unlocking competitive ROI with sustainable AI systems

Secure your spot to stay ahead: https://bit.ly/4mwGngO


At the same time, open source efforts face their own set of hurdles. There has been no scalable infrastructure for collecting the diverse, large-scale data needed to train these agents. Existing open source datasets for graphical user interfaces (GUIs) have limited data, and many research projects provide insufficient detail about their methods, making it difficult for others to replicate their work.

According to the paper, “These limitations collectively hinder advances in general-purpose CUAs and restrict a meaningful exploration of their scalability, generalizability, and potential learning approaches.”

Introducing OpenCUA

OpenCUA framework Source: XLANG Lab at HKU

OpenCUA is an open source framework designed to address these challenges by scaling both the data collection and the models themselves. At its core is the AgentNet Tool for recording human demonstrations of computer tasks on different operating systems.

The tool streamlines data collection by running in the background on an annotator’s personal computer, capturing screen videos, mouse and keyboard inputs, and the underlying accessibility tree, which provides structured information about on-screen elements. This raw data is then processed into “state-action trajectories,” pairing a screenshot of the computer (the state) with the user’s corresponding action (a click, key press, etc.). Annotators can then review, edit, and submit these demonstrations.

AgentNet tool Source: XLang Lab at HKU

Using this tool, the researchers collected the AgentNet dataset, which contains over 22,600 task demonstrations across Windows, macOS, and Ubuntu, spanning more than 200 applications and websites. “This dataset authentically captures the complexity of human behaviors and environmental dynamics from users’ personal computing environments,” the paper notes.

Recognizing that screen-recording tools raise significant data privacy concerns for enterprises, the researchers designed the AgentNet Tool with security in mind. Xinyuan Wang, co-author of the paper and PhD student at HKU, explained that they implemented a multi-layer privacy protection framework. “First, annotators themselves can fully observe the data they generate… before deciding whether to submit it,” he told VentureBeat. The data then undergoes manual verification for privacy issues and automated scanning by a large model to detect any remaining sensitive content before release. “This layered process ensures enterprise-grade robustness for environments handling sensitive customer or financial data,” Wang added.

To accelerate evaluation, the team also curated AgentNetBench, an offline benchmark that provides multiple correct actions for each step, offering a more efficient way to measure an agent’s performance.

A new recipe for training agents

The OpenCUA framework introduces a novel pipeline for processing data and training computer-use agents. The first step converts the raw human demonstrations into clean state-action pairs suitable for training vision-language models (VLMs). However, the researchers found that simply training models on these pairs yields limited performance gains, even with large amounts of data.

OpenCUA chain-of-thought pipeline Source: XLang Lab at HKU

The key insight was to augment these trajectories with chain-of-thought (CoT) reasoning. This process generates a detailed “inner monologue” for each action, which includes planning, memory, and reflection. This structured reasoning is organized into three levels: a high-level observation of the screen, reflective thoughts that analyze the situation and plan the next steps, and finally, the concise, executable action. This approach helps the agent develop a deeper understanding of the tasks.

“We find natural language reasoning crucial for generalizable computer-use foundation models, helping CUAs internalize cognitive capabilities,” the researchers write.

This data synthesis pipeline is a general framework that can be adapted by companies to train agents on their own unique internal tools. According to Wang, an enterprise can record demonstrations of its proprietary workflows and use the same “reflector” and “generator” pipeline to create the necessary training data. “This allows them to bootstrap a high-performing agent tailored to their internal tools without needing to handcraft reasoning traces manually,” he explained.

Putting OpenCUA to the test

The researchers applied the OpenCUA framework to train a range of open source VLMs, including variants of Qwen and Kimi-VL, with parameter sizes from 3 billion to 32 billion. The models were evaluated on a suite of online and offline benchmarks that test their ability to perform tasks and understand GUIs.

The 32-billion-parameter model, OpenCUA-32B, established a new state-of-the-art success rate among open source models on the OSWorld-Verified benchmark. It also surpassed OpenAI’s GPT-4o-based CUA and significantly closed the performance gap with Anthropic’s leading proprietary models.

OpenCUA shows massive improvement over base models (left) while competing with leading CUA models (right) Source: XLANG Lab at HKU

For enterprise developers and product leaders, the research offers several key findings. The OpenCUA method is broadly applicable, improving performance on models with different architectures (both dense and mixture-of-experts) and sizes. The trained agents also show strong generalization, performing well across a diverse range of tasks and operating systems.

According to Wang, the framework is particularly suited for automating repetitive, labor-intensive enterprise workflows. “For example, in the AgentNet dataset, we already capture a few demonstrations of launching EC2 instances on Amazon AWS and configuring annotation parameters on MTurk,” he told VentureBeat. “These tasks involve many sequential steps but follow repeatable patterns.”

However, Wang noted that bridging the gap to live deployment requires addressing key challenges around safety and reliability. “The biggest challenge in real deployment is safety and reliability: the agent must avoid mistakes that could inadvertently alter system settings or trigger harmful side effects beyond the intended task,” he said.

The researchers have released the code, dataset, and weights for their models.

As open source agents built on frameworks like OpenCUA become more capable, they could fundamentally evolve the relationship between knowledge workers and their computers. Wang envisions a future where proficiency in complex software becomes less important than the ability to clearly articulate goals to an AI agent.

He described two primary modes of work: “offline automation, where the agent leverages its broader software knowledge to pursue a task end-to-end,” and “online collaboration, where the agent responds in real-time and works side by side with the human, much like a colleague.” Basically, the humans will provide the strategic “what,” while increasingly sophisticated AI agents handle the operational “how.”



Source link
Continue Reading
Click to comment

You must be logged in to post a comment Login

Leave a Reply

Funding & Investment in Travel

cheap flights to Europe from Melbourne

Published

on


Melbourne’s chilly winter got you craving a getaway? You’re not alone. According to research from the Tourism and Transport Forum Australia, 62 per cent of Aussies are planning to embrace a new travel trend this year.

If you’re among them, and looking for low-cost inspo, we’re here to help. From Wednesday, August 27, Australia’s biggest online mega sale is back with its most popular annual sale: Click Frenzy Travel. Across the four-day sale, you can score huge deals from more than 15 travel providers, with cut-price travel treats ranging from flights to Europe to Disneyland tickets. 

This year’s Click Frenzy Travel will kick off at 7pm on Wednesday, August 27, and run until midnight on Sunday, August 31. By partnering with brands, Click Frenzy streamlines the travel deal hunting mission – bringing a heap of offers together in one place. The full details will only be revealed once the sale goes live, but we’ve got a sneak peek at what’s on offer in this year’s frenzy.

Keen for an overseas escape? You could save a whole lot of money on flights by booking through Click Frenzy; United Airlines, Qatar Airways and Batik Air are all offering bargain flights from Melbourne through the sale.

For a trip to Asia, Batik Air is your best bet – with flights from Melbourne to Bali starting at $279, and flights to Malaysia starting at $399. If you’re craving a European escape, Qatar Airways has got you – with up to 12 per cent off flights to Italy, Greece, France and other Euro summer hot spots.

Cruise enthusiast? Cruise operators Cruiseaway, Fantasea Cruising and iFlyGo are offering deals on cruises around Tasmania, Croatia, Vietnam’s H Long Bay and beyond.

Other top deals in this year’s sale will include 75 per cent off hotels in Vietnam via Traveloka, 25 per cent off a heap of high-energy tours with G Adventures and 15 per cent off car rentals across Australia with Bargain Car Rentals. 

Keen? You can browse and book your next travel adventure through the Click Frenzy website from 7pm on Wednesday, August 27.

Stay in the loop: sign up for our free Time Out Melbourne newsletter for the best of the city, straight to your inbox.

RECOMMENDED:



Source link

Continue Reading

Funding & Investment in Travel

Fintech Klarna Eyeing Possible $14B Valuation In September IPO

Published

on


After hitting the brakes on its IPO in April, Swedish fintech giant Klarna is reportedly resuming its plans to go public in the United States in September, Reuters reported on Tuesday.

The company is believed to be seeking a valuation of between $13 billion and $14 billion. And its shares could be priced at between $34 and $36 as early as this week, according to Reuter’s sources. Klarna is believed to be attempting to raise nearly $1 billion from the initial public offering.

Klarna filed a draft registration statement with the U.S. Securities and Exchange Commission last November, and in March made its F-1 prospectus public. By early April, Klarna seemed to be hitting an indefinite pause on its plans after President Trump announced sweeping tariffs. At its peak, Klarna — which has evolved its model to do more than just buy now, pay later — was valued at $45.6 billion.

More recently, Klarna was valued at $14.6 billion. Since its 2005 inception, the company has raised nearly  $6.2 billion in funding from investors such as Sequoia Capital, General Atlantic and Silver Lake, with Santander adding $1.63 billion to that total in a debt financing just last week. Unlike many other fintechs, Klarna is profitable and turned net income of $21 million in 2024.

In 2025 so far, we have seen an uptick in IPOs compared to recent years. High-performing public debuts this summer from companies like Figma, Circle and Chime Financial helped drive the narrative that the tech IPO market is back in business.

In recent days and weeks, however, shares of many of the most sought-after new market entrants have tumbled. And while by many metrics, valuations still look elevated, the degree has lessened.

Related Crunchbase query:

Related reading:

Illustration: Dom Guzman


Stay up to date with recent funding rounds, acquisitions, and more with the
Crunchbase Daily.



Source link

Continue Reading

Funding & Investment in Travel

Global Investor Jeremy Kranz On Why Not ‘Everything Important Happens In Silicon Valley’

Published

on


Jeremy Kranz left GIC, the Singaporean sovereign wealth fund, in late 2021 after nearly two decades. During his tenure, he served on the boards of DoorDash and Affirm, and was heavily involved with food delivery companies across emerging markets.

An early investor in Zoom, Coinbase and Snowflake, Kranz went on to launch his own venture firm, Sentinel Global, in August 2022 with the goal of “connecting visionary founders with real-world adopters.” In June, Kranz announced the close of the San Francisco-based firm’s inaugural fund, Sentinel Fund I, with committed capital totaling $213.5 million.

During his time at GIC, the most valuable lesson he learned, Kranz said,  “is how emerging markets evolved in innovation capability.”

Jeremy Kranz, managing partner and founder of Sentinel Global

“Twenty years ago, emerging markets were deficient in core innovation. Ten years ago, they became excellent fast followers,” he told Crunchbase News. “By the time COVID happened, emerging markets — mainly China — had become leaders in core innovation, particularly in AI.”

The Chinese, in Kranz’s view, commercialized AI “more effectively and far earlier” than Silicon Valley discovered its true promise.

With Sentinel, Kranz aims to take the lessons he learned during his time at GIC to invest globally in multistage enterprise technology companies.

Kranz describes Sentinel as a multistage venture fund that is thematic in nature. It focuses on three core themes: interoperable commerce; the financial internet, or the “Finternet;” and next-generation enterprise stacks.

In an email interview with Crunchbase News, Kranz shared his vision for Sentinel, why he doesn’t believe every development from OpenAI should be breaking news, and why he thinks that one day IPOs could become nonevents.

The interview has been edited for brevity and clarity.

What would you say are the most valuable lessons you learned from your time at GIC? What were some of the most notable investments you were involved in?

Besides how emerging markets evolved in innovation capability, I learned that it’s important to remain rational through market cycles. There are market cycles where you’re trying to be pragmatic and rational in environments that are fundamentally crazy — either so bullish that pricing defies belief, or so negative that no deals get done and innovation seems to have stopped. You have to maintain good grounding and interpret who’s operating from fear versus who brings clarity of purpose, underwriting and vision.

My most notable investments centered around what I call “the movement of people and packaging of food.” I’ve been passionate about food delivery since childhood and consider myself an expert in this space. Over my 25 years in VC, I followed this trend line from early losses with Webvan (a dot-com grocery delivery company) to investing in food delivery during a major down market — companies like DoorDash and Uber in the U.S., Meituan in China, and Flipkart, Ola, Grab, Souq and Rappi in other emerging markets.

These evolved into platform companies, not just delivery companies, expanding into payments and other services. … Most companies I backed are now over 10 years old and have become the incumbents that the next generation is targeting.

Tell us more about Sentinel. What is your average check size? Who are your LPs?

We typically invest in Series A, B, and C rounds, writing checks ranging from single digits to mid- to high-double digits (in millions). Our LP base includes prominent sovereign wealth funds and family offices that are available to partner with us as co-investors on deals.

What we look for is the ability to leverage our network outside the U.S. to help companies go global. We call this “Sentinel Labs” — it’s the continuation of work I did at GIC with the Bridge Forum I founded, which was a platform connecting enterprises outside the U.S. with startups in developed markets.

Why do you think China’s AI tech is ahead of the U.S.? How has that allowed China to infiltrate the U.S. economy? What are U.S. investors still missing?

The Chinese are exceptionally smart about commercializing technology. Years ago, pre-COVID, I visited ByteDance‘s R&D labs. After visiting labs at Microsoft, Google and other great tech companies, I know what American Silicon Valley companies with infinite R&D budgets look like, often tinkering on quantum computing and other research without clear commercialization paths.

But at ByteDance, scientists are responsible for both inventing and commercializing. During my full-day visit, they showed me not just basic research, but demos of technologies they were actively commercializing. The pathway from R&D to commercialization was maintained tightly; they had to show results within a year.

This approach allowed companies to successfully infuse AI long before Silicon Valley popularized the idea of a new AI Industrial Revolution.

TikTok exemplifies this perfectly. TikTok’s success wasn’t due to better content or superior marketing to kids. It was simply smarter at using AI to make content more attractive, addictive and engaging. Its algorithm for curating user content is its unique value proposition, predicated on extremely effective AI that analyzes user signals to determine optimal daily content.

DJI provides another compelling example. Many drone companies in Silicon Valley had substantial funding and talent, but couldn’t make drones fly long enough, carry sufficient weight, or avoid obstacles. DJI built what I consider the world’s greatest consumer drone by leveraging AI and recognizing that features like sonic collision avoidance required purpose-built semiconductors. DJI partnered with the Chinese government to develop semiconductor processes, enabling them to employ and commercialize AI in drones with unmatched results.

The contrast is striking: In the U.S., we found ourselves stuck in labs with clipboards and lab coats, essentially waiting for breakthroughs to happen.

Today, Americans are inventing and commercializing applications across various technologies, particularly LLMs, which appears to be effective catch-up. In some areas, we might be leaping forward.

However, I find it concerning that U.S. media has a celebrity-obsessed approach. Every development from OpenAI becomes breaking news, creating the impression that everything important happens in Silicon Valley. I absolutely disagree with this narrative.

I’m confident that at this very moment, the Chinese have invented and commercialized AI that is not only globally competitive but arguably more effective for specific applications. They will export these technologies. We must be careful not to let the loudest environment be viewed as the most successful.

While I’m proud of America’s AI leadership and expect continued leadership, we cannot be overly self-centered. We must remain deferential to the fact that China has a long history of inventing and commercializing AI before Silicon Valley. Given this track record, it’s hard to believe they’ve suddenly fallen behind.

The media hype around the Valley needs to be balanced with realistic understanding of the past 15 years in artificial intelligence development globally.

Where are the next great tech IPOs (really) coming from? Why does today’s AI boom hinge “on the ‘boring’ infrastructure layer no one’s covering?”

At Sentinel, we hold a controversial belief about the future of IPOs. We think the current administration is blazing a trail of tokenization across all asset classes. Right now, we’re seeing tokenization for cash — the most liquid asset in the world got more liquid. While it seems odd to make cash more liquid through tokenization, there are genuine benefits.

This began with the GENIUS (Guiding and Establishing National Innovation for U.S. Stablecoins) Act. The next development will be the Clarity Act, which we expect will enable tokenization of public stocks, private companies and private credit. The experimental possibilities are extensive.

When this happens, IPOs will become one liquidity option among many, but may not be the highest priority for all companies. Today’s secondaries market is booming but restricted — you must be a registered securities buyer, and transactions are largely one-to-one. You can’t simply purchase company shares on a platform like eBay.

We envision a world where the Clarity Act and tokenization of real-world assets dramatically transform how private markets raise money and seek liquidity. I’d call this not one black swan event, but possibly 10 black swans. We’re entering an era where traditionally illiquid asset classes may become significantly more liquid.

This shift will fundamentally change the nature and importance of IPOs. For some companies, IPOs could become nonevents because public-market investors will have already accessed tokenized versions of those shares before the IPO. The IPO becomes a less significant milestone in a company’s lifecycle.

How do you believe the Trump administration is lowering friction in the capital markets, and what does that mean for the future of venture capital investing?

The capital market changes won’t necessarily impact venture investing first. The transformation will likely begin with digital cash, then extend to public stocks and private credit, with private companies coming later in the sequence.

The key development is tokenization of real-world assets, which features two particularly innovative elements for capital markets.

First, smart contracts can embed information and validation directly into transactions. KYC (Know Your Customer) requirements can be built into token transactions, significantly reducing friction and costs for market changes.

Second, and more controversially, is enabling yield-based transfers of cash or tokenized money market funds as payment methods. This concept is potentially transformative.

Currently, we deposit cash in banks that provide roughly 2.5% returns even when Treasuries yield 5%, because banks capture the difference while risking our deposits in other assets. With blockchain-based saving accounts tied to Treasuries, I should receive nearly the full 5% yield.

The revolutionary aspect: if I can use these tokenized treasury-linked assets for payments, every transaction transfers yield rights along with the principal. When I Venmo you $10, I’d transfer the rights to the yield on that $10. This would be both exciting and terrifying for global capital markets.

There’s a scenario where this experiment could be given life through innovation-friendly regulation. This represents a major debate point for the Clarity Act. While the GENIUS Act sidestepped this issue, the Clarity Act will address it directly.

Companies like Circle already provide rewards for USDC that could be perceived as yield, but it’s structured as token rewards rather than direct U.S. dollar yield. We’re just one step away from explicit dollar-based yield on cash that can be used for payments and transfers.

This is the major black swan event I believe is approaching.

Illustration: Dom Guzman


Stay up to date with recent funding rounds, acquisitions, and more with the
Crunchbase Daily.



Source link

Continue Reading

Trending

Copyright © 2025 AISTORIZ. For enquiries email at prompt@travelstoriz.com