AI Industry Updates, June 2024: On-Device Processing

Published 25 June 2024

Author
Adam Speight
2 min read

While cloud-based artificial intelligence (AI) tools, like ChatGPT, have gained popularity in 2024, on-device AI offering enhanced privacy, performance and personalisation has lagged. In the past month, technology manufacturers have revealed hardware and software that allows AI processing on user devices.

  • Chip Manufacturers Bring AI Power: Joining US-based chipmaker Qualcomm with its previously announced Snapdragon X series, American peers Intel and AMD revealed new consumer offerings – Lunar Lake and Ryzen AI 300 chips – to power Windows PCs with onboard AI at B2B technology trade show Computex Taipei 2024.

    These products all have one thing in common: a neural processing unit (NPU) featuring 40-plus trillion operations per second (TOPS). This level of NPU processing power enables new on-device AI features, like real-time creative assistance and enhanced translation (see Microsoft's Copilot+ PCs Debut New AI Features).

  • Apple Integrates Privacy-Focused AI Features: At its Worldwide Developers Conference 2024, Apple revealed Apple Intelligence for iPhone, iPad and Mac. Apple’s Siri has been enhanced, using on-device data to provide more contextual and personalised responses. Apple Intelligence introduces writing assistance, image generation, email organisation and call summarisation features, leveraging improved natural language understanding and on-screen awareness.

    For complex requests, the company will still go off-device. To protect user privacy, Apple has developed a new Private Cloud Compute feature. This uses Apple’s silicon-powered servers, with user data never stored and only deployed for users’ own requests.

    Apple has partnered with OpenAI’s ChatGPT for experiences where Siri requires “broad world knowledge”, and the chatbot is integrated into system-wide writing tools. Siri will ask users for permission to communicate with ChatGPT. Users are not required to log in to ChatGPT, their IP addresses are obscured and OpenAI won’t store requests.