The U.S. Patent and Trademark Office has awarded Apple U.S. Patent No. 12,353,649, indicating the company’s aspirations go beyond the iPad for the Apple Pencil’s future. This new patent reveals significant advancements in stylus technology.
According to the patent filing, titled “INPUT DEVICE WITH OPTICAL SENSORS” (via Patently Apple), Apple is exploring a stylus integrated with optical sensors capable of tracking motion, orientation, and positioning in three-dimensional space, all without physical contact with a screen. This capability could facilitate drawing on nearly any surface or even in the air within a limited range.
This concept might seem familiar; current Apple Pencils can already sense tilt and pressure. Additionally, during WWDC25 just last month, Apple introduced the Logitech Muse, a spatial stylus for the Vision Pro, enabling users to sketch mid-air.
The patent outlines different techniques for drawing without direct touch. It describes two categories of optical sensors that could be incorporated into the stylus:
– **Optical flow sensors**, functioning similarly to optical mice by examining frame-by-frame changes in light to detect movement.
– **Laser speckle flow sensors**, monitoring granular shifts in laser reflections as the stylus moves.
Both sensor types are capable of detecting motion even when hovering up to 100mm (or 3.94 inches) above a surface, while allowing the stylus to interact with non-touch-sensitive materials such as wood or paper.
Additionally, one model of the Pencil detailed in the patent includes a trackball tip that rotates as the stylus is moved. Instead of relying on mechanical encoders like traditional mice, Apple proposes the use of internal optical sensors to monitor the trackball’s movement.
As articulated in paragraph 64 of the patent, “In some examples, optical sensor(s) are disposed within a portion of a stylus body (e.g., housing of a stylus) separate from the stylus tip. The optical sensor(s) can be configured to measure displacement of internal components of a stylus, such as by tracking displacement of a trackball at least partially disposed in the housing of the stylus. (…) Although disposing the trackball at the stylus tip prevents the optical sensor(s) from accurately measuring displacement with respect to the z-axis, tilt, orientation, and/or axial rotation of the stylus, disposing the trackball at the stylus tip causes the surface to be less prone to scratch and contamination, to provide improved signal-to-noise ratios when detected by the optical sensor(s), and to be agnostic to surface optical properties.”
The patent also hints at integrating this technology with additional sensors, including inertial measurement units, force detectors, and machine vision. Such enhancements could facilitate gesture input, handwriting recognition, and 3D drawing capabilities.
This innovation positions the Apple Pencil as a fitting accessory for devices like the Vision Pro, where user input extends beyond traditional touchscreen interactions, showcasing Apple’s ongoing commitment to refining its vision for the spatial computing age.
