OpenAdapt

What is OpenAdapt?

OpenAdapt is an open-source AI-first process automation system that learns to automate tasks in desktop and web applications by observing human demonstrations. It captures user interactions, including mouse, keyboard, and UI element data, and uses Visual Language Models to automate workflows, emphasizing privacy-sensitive data handling. Its goal is to reduce repetitive tasks, offering a simpler user interface for knowledge workers.

Key features

• Records screenshots and associated user input.
• Aggregates and visualizes user input and recordings.
• Converts screenshots and user input into tokenized formats.
• Generates and replays synthetic input via transformer models.
• Creates process graphs from recording logs.
• Supports all desktop GUIs, including virtualized and web.
• Provides state-of-the-art GUI understanding with Segment Anything.
• Includes privacy-focused PII/PHI scrubbing
• Enables decentralized, secure data sharing
• Detailed performance monitoring.
• System tray icon and web dashboard.

Use cases

• Corporation Registration
• Freight/logistics
• Insurance underwriting + loss adjusting
• Affiliate Marketing
• Paralegal
• Human genetics
• Pharmacist
• Options Trader
• Physician Documentation
• Recruiter
• Real Estate Agent
• Product Manager