Invented by Julien Y. Robert, Tyler M. Fox, Camille Moussette, Viktor Miladinov, Jean-Pierre M. Mouilleseaux, Madeleine S. Cordier, Apple Inc
Tactile feedback, also known as haptic feedback, is the sensation of touch or vibration that is generated by a device in response to user input. It adds a new dimension to the user experience, allowing users to feel and interact with digital content in a more realistic and intuitive way.
One of the key drivers behind the growing market for tactile output devices is the increasing demand for virtual reality (VR) and augmented reality (AR) applications. These technologies rely heavily on visual and auditory inputs to create immersive experiences, but the addition of tactile feedback can greatly enhance the overall experience. For example, in a VR game, users can feel the impact of a virtual punch or the texture of a virtual object, making the experience more lifelike.
There are several devices and methods available in the market that enable tactile outputs using graphical user interfaces. One popular device is the haptic feedback glove. These gloves are equipped with sensors and actuators that can simulate the sensation of touch and pressure on the user’s hands. They can be used in various applications, such as gaming, virtual training, and medical simulations.
Another device gaining traction in this market is the haptic feedback vest. These vests are designed to provide tactile feedback to the user’s torso, allowing them to feel vibrations and movements in virtual environments. They are particularly useful in applications like virtual concerts, where users can feel the bass of the music or the crowd’s energy.
In addition to these devices, there are also software-based methods for generating tactile outputs. For example, some smartphones and tablets are equipped with vibration motors that can provide haptic feedback when interacting with the device’s touchscreen. This allows users to feel a subtle vibration when typing on a virtual keyboard or navigating through menus.
The market for devices and methods for generating tactile outputs using graphical user interfaces is expected to continue growing in the coming years. The advancements in technology, such as the development of more compact and affordable haptic feedback devices, will drive the adoption of this technology in various industries.
The gaming industry, in particular, is expected to be a major driver of growth in this market. With the increasing popularity of VR and AR gaming, the demand for more immersive experiences will fuel the need for tactile feedback devices. Additionally, industries like healthcare, automotive, and education are also expected to adopt this technology to enhance training simulations, improve user interfaces, and create more realistic experiences.
In conclusion, the market for devices and methods for generating tactile outputs using graphical user interfaces is experiencing significant growth. The demand for more immersive and engaging experiences in virtual and augmented reality applications is driving the adoption of haptic feedback devices. As technology continues to advance, we can expect to see more innovative solutions in this market, making digital interactions more lifelike and interactive.
The Apple Inc invention works as follows
An electronic display of a user interface” The electronic device detects the first user interaction through the touch-sensitive display while displaying the interface and one or more tactile generators are in a reduced power state. The electronic device, in response to the detection of the first interaction by the user, sets one or more tactile generators into a low latency state. After setting one or more tactile input generators to a low-latency mode, the electronic device detects the second user interaction, which is a continuation of the first user’s touch input on the touch sensitive surfaces. The electronic device, upon detecting the second interaction of the user, generates a tactile response that corresponds with the second interaction.
Background for Devices and methods for generating tactile outputs using graphical user interfaces
In recent years, the use of touch-sensitive surfaces for input devices on computers and other electronic computing device has grown significantly. Touchpads and touchscreen displays are examples of touch-sensitive surfaces. These surfaces are used widely to manipulate objects and user interfaces on a display. Examples of user interface objects are digital images, videos, text, icons and control elements like buttons and other graphics.
The use of haptic feedback in conjunction with audio and/or visual feedback is a common way to improve the operability of electronic gadgets.
However applications that respond to the same gesture with different haptic feedback present challenges for users and may result in unintended actions. The user must cancel unintended actions by the device and then provide inputs once again. These operations are tedious and cumbersome. It takes longer to perform unintended actions and inputs, which wastes energy. This last consideration is especially important for battery-operated gadgets. It would be nice to have a framework that provides haptic feedback.
There is a requirement for electronic devices that provide haptic feedback faster, more efficiently, and with more consistency (usually in conjunction with audio and/or visual feedback), making the manipulation of user interfaces easier and more intuitive for an end-user. These methods and interfaces can be used to complement or replace existing methods of providing haptic input. These methods and interfaces allow users to reduce the amount, nature, or extent of inputs by assisting them to understand the relationship between inputs provided and the device’s response to those inputs. This creates a more efficient interface. These methods and interfaces also provide a consistent user interface across a variety of software applications. Such methods and interfaces are particularly useful for battery-operated gadgets, as they conserve energy and extend the time between batteries charges.
The disclosed devices reduce or eliminate the above deficiencies as well as other problems related to user interfaces of electronic devices with touchscreen surfaces. In some embodiments the device is a computer. In some embodiments the device can be portable (e.g. a laptop computer, tablet PC, or handheld device). In certain embodiments, it is a personal device (e.g. a watch or wearable electronic device). In some embodiments the device includes a touchpad. In certain embodiments, the display is touch-sensitive (also called a “touch screen”). or ?touch-screen display?). In some embodiments the device includes a graphical interface (GUI), a processor, memory, and modules, programs, or sets of instructions that are stored in the memory to perform multiple functions. In some embodiments the GUI is primarily manipulated by the user using a stylus or finger on a touch-sensitive surface. In some embodiments the functions may include drawing, image editing, word processing and spreadsheet creation, as well as game play, telephoning or video conferencing, instant messaging, e mailing, digital photography, digital videoing, internet browsing, digital music, notes, and/or video playing. The executable instructions to perform these functions can be included, optionally, in a nontransitory computer-readable storage medium, or another computer program product configured to execute by one or multiple processors.
According to some embodiments, the method is performed on an electronic device that has a display, input devices and tactile output generators. Displaying a user interface of a first application on the display is part of the method. The user interface includes a plurality of elements to which user interaction models from a plurality of user interaction models provided by an application-independent module have been assigned, the plurality of elements including a first element to which a first user interaction model has been assigned, the plurality of elements having content provided by an application-specific module for the first software application. The first user-interaction model describes how the user interface reacts to inputs directed at the first element of a user interface. While displaying the interface, the method includes detecting an input directed at the first element. In response to detecting an input, the method includes updating the user interface displayed on the display according to characteristics of the entered data; if the input is determined to meet tactile output criteria specified in the first user interactivity model, generating the first tactile output; and if the input is determined not to meet tactile output criteria specified in the first user interactive model, forgoing the generation of the initial tactile output.
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and one of more tactile output generators. Displaying a user interface on the display and detecting a first event associated with a tactile input are part of the method. The first user interface event is generated by displaying changes to the user-interface. For a certain amount of time, the device won’t be able generate the tactile output that is associated with the first event. In response to detecting occurrence of a first condition and in accordance with a decision that the first event of user interaction corresponds to a category of first events of user interaction, the method includes delaying the generation of the event of first user contact for at the least the respective time. After delaying the event of first user contact for at the least the respective time, the method displays the one or multiple changes to the interface and generates the tactile output associated with the event of first user connection. The display of one or more changes in the user interface occurs simultaneously with the generation the tactile output associated with the first interface event.
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and one of more tactile output generators. While the one or multiple tactile output generators is in a low power state, the method comprises receiving an indication of a user interface. The tactile output is assigned to a portion of the interaction. In the low-power mode, tactile outputs will be generated with a certain amount of latency. In response to the indication that a user interaction associated with the tactile input has begun, the method includes setting one or more tactile generators into a low latency state for the first time, if the indication is determined to meet tactile output generator readiness criteria. The low-latency is a state where the tactile outputs will be generated with a lower amount of delay than the initial amount. After setting up the one-or-more tactile outputs in the low latency state, the method includes: if the user interaction reaches the portion of the interaction associated with the tactile out put before a predefined time period since the initial time period has passed and the one-or-more tactile outputs generators are in the low latency state, the method also includes generating the output using one-or-more tactile outputs generators in the second amount.
According to some embodiments, an electronic device having a display, touch-sensitive surfaces, and one or multiple tactile output generators performs a method. The method includes, while a first type of event is associated with a first tactile output and a second type of event that is distinct from the first type of event is associated with a second tactile output that is different from the first tactile output by an application-independent module, receiving first information from a first application; and, in response to receiving the first information from the first application, generating a response to the first information, including: in accordance with a determination that the first information corresponds to an application event of the first type, generating the first tactile output using the one or more tactile output generators; and, in accordance with a determination that the first information corresponds to an application event of the second type that is distinct from the first type, generating, using the one or more tactile output generators, the second tactile output that is different from the first tactile output. After generating the response, the method includes receiving second data from a different application from the first one. The response is generated by generating, using one or multiple tactile outputs, the first tactile output.
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and one, two, or three tactile output generators. The method includes receiving, at an application-independent module, from an application-specific module that is associated with a first application, information about an input directed to a user interface of the first application. The application-specific module specifies a tactile pattern for an operation that is performed in the user’s interface of the first app in response to the input. Information about the input may include information that indicates the magnitude of an operation performed on the user interface as a result of detecting the input. In addition, the method includes, upon receiving information about the input directed at the user interface for the first application, creating, via one or more tactile out put generators, an output tactile that corresponds with the operation performed on the user interface for the first app. The tactile output is a pattern of tactile output specified by the module application-specific. The amplitude of the tactile output is determined by the magnitude of the action performed in the user interface in the first application as a response to the input.
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and one or multiple tactile output generators. Displaying a user interface is part of the method. Detecting a first interaction by the touch-sensitive surfaces while the tactile output generators in low-power are also part of the method. In response to the first interaction, the method includes setting one or multiple tactile outputs generators into a low latency state. After setting one or several tactile outputs generators into the low-latency, the method includes detecting a subsequent user interaction which is part of the same sequence as the initial user interaction. In response to detecting a second user interface, the method includes generating a touch output that corresponds with the second interaction.
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and/or tactile output generators. Receiving a request for a user interface operation is part of the method. The first user-interface operation is a change of a particular user-interface element associated with a tactile output. In response to the first request, the method includes performing the first interface operation and producing the first tactile output. After performing the first user-interface operation, the method includes receiving a subsequent request for a secondary user-interface operation which is associated to a third tactile output. In response to receiving this second request, if it is determined that the request is for an instance of a first type change in a user-interface element that was associated with the third tactile output, the method comprises: “determining a period of time between a previous instance of a first type change in a respective user element for which tactile outputs
According to some embodiments, the method is performed on an electronic device that has a display, touch-sensitive surfaces, and/or one or more tactile output devices. The method includes, at an application-independent module, receiving user interface information from an application. The user interface information is a set of one or multiple displayed user interfaces with one or several dimensions defined by the application-specific module. The method also includes receiving an input directed toward one or more of the displayed user interface elements; and, at the application-independent module, determining one or more tactile outputs to be generated based on a magnitude of the input and the one or more dimensions defined by the applications-specific module; and, using the one or more tactile output generators, generating the determined one or more tactile outputs.
According to some embodiments, a device electronic includes a display module configured to display a user interface; one of more input devices configured to receive inputs and one or multiple tactile output generator modules configured to produce one or several tactile outputs. The display module, one or many input device units and one or several tactile output generator modules are all coupled to a processing unit. The processing unit is designed to display a user interface of a first software app on the display unit. The user interface includes a plurality of elements to which user interaction models from a plurality of user interaction models provided by an application-independent module have been assigned, the plurality of elements including a first element to which a first user interaction model has been assigned, the plurality of elements having content provided by an application-specific module for the first software application. The first user-interaction model describes how the user interface reacts to inputs directed at the first element of a user interface. While displaying the user’s interface, the processing unit can detect an input to the first element via one or more input devices units. It then updates the interface on the display based on input characteristics. If the input is determined to meet tactile output criteria defined by the 1st user interaction model it will enable the generation of a tactile output.
According to some embodiments, a device electronic includes a display module configured to display a user interface; a touch sensitive surface unit configured for touch inputs; or one or multiple tactile output generator modules configured to produce one or several tactile outputs. The display unit is coupled to the touch-sensitive unit and to the tactile output generator unit. The processing unit is configured: to enable display of a user-interface on the display unit; and to detect an occurrence a of a condition that triggers first user interface events that are associated with tactile outputs. The first user interface event is generated by enabling the display of changes to the interface. For a predetermined period of time, the device is not able to produce the tactile output that corresponds with the first event. In response to detecting a first condition and a determination that a first interface event belongs to a first interface event category the processing unit is configured to enable the generation of tactile output associated with the initial interface event, delay the generation of the event by at least a respective amount, and then enable the display of one or more changes in the user interface after the delay. The display of one or more changes in the user interface occurs simultaneously with the generation the tactile output associated with the first event.
According to some embodiments, a device electronic includes: a display unit configured for displaying one or multiple user interfaces, a touch-sensitive unit configured to receive inputs from the touch; one or several tactile output units configured to produce one or many tactile outputs and a processor unit coupled to the display unit and the touch-sensitive unit. While the tactile output generators are in low-power mode, the processing unit receives an indication of a user interaction. The tactile output is assigned to a portion of the interaction. In the low-power mode, tactile outputs will be generated with a certain amount of latency. In response to the processing unit receiving an indication that the user’s interaction with the tactile output is underway, and if the indication is determined to meet tactile output generator readiness criteria, the processing unit can also be configured to set one or more tactile generator units into a low latency state for the first time. The low-latency is a state where the tactile outputs will be generated with a lower amount of delay than the initial amount. After setting up the one-or-more tactile output generation units in the low latency state, the processing unit can be configured to: if the user interaction reaches the portion of the interaction associated with the tactile out put before a certain amount of time from the first time elapsed, and if the one-or-more tactile output generation units are still in low-latency, then enable the generation of the tactile input using the one-or-more tactile output generating units with a second amount of latency;
The processing unit is configured to, while a first type of event is associated with a first tactile output and a second type of event that differs from the first type of event is associated with a second tactile output that’s different from the initial tactile output by an application-independent module: receive the information received in the form of a user interface; a touch sensitive surface unit configured for receiving touch inputs; and one or multiple tactile output generation units configured to produce one or several tactile outputs. The processing unit is configured to, while a first type of event is associated with a first tactile output and a second type of event that is distinct from the first type of event is associated with a second tactile output that is different from the first tactile output by an application-independent module: receive first information from a first application; and, in response to receiving the first information from the first application, enable generation of a response to the first information, including: in accordance with a determination that the first information corresponds to an application event of the first type, enable generation of the first tactile output using the one or more tactile output generator units; and, in accordance with a determination that the first information corresponds to an application event of the second type that is distinct from the first type, enable generation, using the one or more tactile output generator units, of the second tactile output that is different from the first tactile output. After enabling the generation of response to first information, the processing unit is configured to receive second data from a different application from the first one. The second data is then used to enable the generation of response to second information.
According to some embodiments, a device electronic includes a display module configured to display one user interface; a touch sensitive surface unit configured for touch inputs; or one or multiple tactile output generators configured to produce one or several tactile outputs. The display unit is coupled to the touch-sensitive unit and to the tactile output generators. The processing unit is configured to receive, at an application-independent module, from an application-specific module that is associated with a first application, information about an input directed to a user interface of the first application. The application-specific module specifies a tactile pattern for an operation that is performed on the user interface in the first application as a result of detecting the input. Information about the input may include information regarding the magnitude of an operation performed by the user interface as a result of detecting the input. In response to receiving information about input directed at the user interface for the first application the processing unit is configured to enable the generation of a tactile out put that corresponds with the operation performed on the user interface for the first applicaiton via one or more tactile generator units. The tactile output is a pattern of tactile output specified by the module application-specific. The amplitude of the tactile output is determined by the magnitude of the action performed in the first application’s user interface in response to the input.
The processing unit is configured to: enable display of a user interface on the display unit; while enabling display of the user interface on the display unit and while the one or two tactile output units are in a low-power state, detect a first interaction via the touch-sensitive unit; as a response to detecting the first interaction, set the one or two tactile output units to a low-latency state; after setting the one-or-more tactile output units to the low-latency state, detect a second user The processing unit is configured: to enable display of a display unit user interface; to detect a first interaction by a user via the touch-sensitive unit unit while in a lower-power state of the tactile output units; to set the tactile output units into a low latency state in response to this first interaction; to detect a subsequent interaction in the same sequence as the first interaction and to enable generation of tactile outputs that correspond to that second interaction.
According to some embodiments, a device electronic includes a display module configured to display a user interface; a touch sensitive surface module configured to receive inputs via touch; one or multiple tactile output generator modules configured to produce one or several tactile outputs and a processor unit connected to the display, touch sensitive surface and tactile output generator modules. The processing unit receives a request for a user interface operation. The first user-interface operation is a change to a user-interface element associated with a tactile output. After receiving the request, the processing unit can perform the first interface operation to enable the generation of the tactile output. It then receives a request for a subsequent user interface that is associated to a tactile output. In response to receiving a second request and based on a determination of whether the request is for a first type change to the user interface associated with a secondary tactile output, the processing unit is configured to: “determine a time gap between a previous instance of a first type change to the respective element of the user interface for which a tactile result was generated and a subsequent instance of that first type change requested by the request. If the time gap is less than or equal to a predefined period of time, the
According to some embodiments, a device electronic includes a display module configured to display a user interface; a touch sensitive surface unit configured for touch inputs; or one or two tactile output generators configured to produce one or multiple tactile outputs. The display unit is coupled to the touch sensitive surface unit and to the one or several tactile output generators. The processing unit is configured to receive, at an application-independent module, user interface information from an application. The user interface information is a set of one or multiple displayed user interfaces with one or several dimensions defined by the application-specific module. The processing unit is also configured to receive an input directed toward one or more of the displayed user interface elements; determine, at the application-independent module, one or more tactile outputs to be generated based on a magnitude of the input and the one or more dimensions defined by the applications-specific module; and enable generation of the determined one or more tactile outputs using the one or more tactile output generator units.
Click here to view the patent on Google Patents.
Leave a Reply