On August 28, 2017, Nintendo Co., Ltd. filed in the US via the United States Patent and Trademark Office a patent application for an INFORMATION PROCESSING SYSTEM, EXTENDED INPUT SYSTEM, AND INFORMATION PROCESSING METHOD. This patent concerns extended input devices for use with Nintendo Switch. Examples are Donkey Konga drums and a keyboard. Inventor is Hitoshi Yamazaki who has seventeen other patents with Nintendo (some together with Shigeru Miyamoto). Below is abstract, selected excerpts and of course drawings.
United States Patent Application 20180065035
Kind Code A1
YAMAZAKI; Hitoshi March 8, 2018
INFORMATION PROCESSING SYSTEM, EXTENDED INPUT SYSTEM, AND INFORMATION PROCESSING METHOD
Abstract
In a state where an extended input device is secured to a data transmission device, an image capturing unit of the data transmission device captures an image of a light emitting unit that emits light, an image of which is allowed to be captured by the image capturing unit in accordance with a user input and/or a user operation, and data generated based on at least the captured image is transmitted to an information processing apparatus. The information processing apparatus executes an information process based on the transmitted data.
Inventors: YAMAZAKI; Hitoshi; (Kyoto, JP)
Applicant:
Name City State Country Type
NINTENDO CO., LTD.
Kyoto
JP
Family ID: 1000002874791
Appl. No.: 15/687771
Filed: August 28, 2017
Claims
1. An information processing system, comprising: an information processing apparatus executing an information process; a data transmission device transmitting data to the information processing apparatus; and an extended input device secured to the data transmission device; wherein: the data transmission device includes: an image capturing unit; and a computer transmitting, to the information processing apparatus, at least data generated based on an image captured by the image capturing unit; the extended input device includes: a securing portion secured to the data transmission device 4; an operation input portion usable by a user to make a user input and/or a user operation for the information process; and a light emitting unit located such that an image thereof is allowed to be captured by the image capturing unit, the light emitting unit emitting light, an image of which is allowed to be captured by the image capturing unit in accordance with the user input to, and/or the user operation on, the operation input portion, in a state where the light emitting unit is secured to the data transmission device; and the information processing apparatus includes a computer receiving the transmitted data and executing the information process based on the received data.
2. The information processing system according to claim 1, wherein: the operation input portion includes a plurality of operation input members; and the light emitting unit includes a plurality of light emitting members, and at least one of the plurality of light emitting members, corresponding to at least one of the operation input members subjected to the user input and/or the user operation, emits light.
3. The information processing system according to claim 2, wherein: the computer of the data transmission device acquires coordinate information on a portion having a high luminance in the image captured by the image capturing unit; in the transmitting of the data to the information processing apparatus, the acquired coordinate information is transmitted as the data generated based on the image captured by the image capturing unit; and in the executing of the information process, it is specified which one of the plurality of light emitting members is emitting light based on the received data, and the information process is executed based on the user input to, and/or the user operation on, the operation input member corresponding to the light emitting member that is emitting light.
4. The information processing system according to claim 2, wherein: the image capturing unit is an infrared camera; and the plurality of light emitting members are a plurality of infrared LEDs.
5. The information processing system according to claim 4, wherein the plurality of infrared LEDs are each located at any of positions arrayed in a matrix on a predetermined surface.
6. The information processing system according to claim 5, wherein the predetermined surface is located inclined with respect to an image capturing direction of the image capturing unit, in a state where the predetermined surface is secured to the data transmission device,
7. The information processing system according to claim 1, wherein: the information processing apparatus is a game apparatus main body; the data transmission device is a game controller including an operation unit; and in the transmitting of the data to the information processing apparatus, data including at least data from the operation unit of the data transmission device and the data generated based on the image captured by the image capturing unit is transmitted.
DrawingsDescription
FIELD
[0002] The technology shown herein relates to an information processing system, an extended input device, and an information processing method, and more specifically, an information processing system including an information processing apparatus, a data transmission device and an extended input device secured to the data transmission device, the extended input device in the information processing system, and an information processing method.
BACKGROUND AND SUMMARY
[0003] Conventionally, there is an accessory device connectable to a mobile device to add a function to the mobile device. For example, there is a technology for connecting, to a connector provided in a controller of a apparatus, another controller as an accessory device. With such a technology, two controllers are usable to provide a wider variety of game operations.
[0004] However, the above-described accessory device includes a circuit usable for communicating with the mobile device and a circuit usable for an information process of generating information to be transmitted to the mobile device. Therefore, there is room for improvement, more specifically, for simplification of the structure of the accessory device.
[0005] Therefore, it is an object of an exemplary embodiment to provide an information processing system realized by a simple structure, an extended input device, and an information processing method.
[0006] In order to achieve the above-described object, the exemplary embodiment may employ, for example, the following configurations. It is understood that, in order to interpret the claims, the scope of the claims should be interpreted only by the recitations of the claims. If there is a contradiction between the recitations of the claims and the descriptions of the specification, the recitations of the claims take precedence.
[0007] An example of information processing system in the exemplary embodiment includes an information processing apparatus executing an information process; a data transmission device transmitting data to the information processing apparatus; and an extended input device secured to the data transmission device. The data transmission device includes an image capturing unit; and a transmission section. The transmission section transmits, to the information processing apparatus, at least data generated based on an image captured by the image capturing unit. The extended input device includes a securing portion; an operation input portion; and a light emitting unit. The securing portion secures the extended input device to the data transmission device. The operation input portion is usable by a user to make a user input and/or a user operation for the information process. The light emitting unit is located such that an image thereof is allowed to be captured by the image capturing unit, and emits light, an image of which is allowed to be captured by the image capturing unit in accordance with the user input to, and/or the user operation on, the operation input portion, in a state where the light emitting unit is secured to the data transmission device. The information processing apparatus includes a receiving section and a processing section. The receiving section receives the data transmitted from the transmission section. The processing section executes the information process based on the data received by the receiving section.
[0008] According to the above, the extended input device is attached to the data transmission device that captures an image of the light emitting unit that emits light in accordance with the user input and/or the user operation. By use of the data generated based on the captured image of the light emitting unit, the information process is executed. Therefore, the extended input device does not need to include an electric component that transmits the result of the detection of the user input or the user operation to the information processing apparatus. For this reason, the structure of the extended input device is simplified. As compared the case where the captured image itself is transmitted from the data transmission device to the information processing apparatus, the amount of data to be transmitted is decreased.
[0009] The operation input portion may include a plurality of operation input members. The light emitting unit may include a plurality of light emitting members, and at least one of the plurality of light emitting members, corresponding to at least one of the operation input members subjected to the user input and/or the user operation, may emit light.
[0010] According to the above, the light emitting unit that is emitting light is detected, so that the operation input member subjected to the user input and/or the user operation is distinguished.
[0011] The data transmission device may further include an image processing section. The image processing section acquires coordinate information on a portion having a high luminance in the image captured by the image capturing unit. In this case, the data transmission device may transmit the coordinate information acquired by the image processing section, as the data generated based on the image captured by the image capturing unit. The processing section may specify which one of the plurality of light emitting members is emitting light based on the data received by the receiving section, and execute the information process based on the user input to, and/or the user operation on, the operation input member corresponding to the light emitting member that is emitting light.
[0012] According to the above, an image of the light emitting unit that emits light in accordance with the user input and/or the user operation is captured, so that the user input to, and/or the user operation on, the extended input device is distinguished. Therefore, various types of extended input devices may be added by use of the same predetermined data format with no need to prepare a new data format.
[0013] The image capturing unit may be an infrared camera. The plurality of light emitting members may be a plurality of infrared LEDs.
[0014] According to the above, an image of the infrared light emitted by the infrared LED is captured by the infrared camera. Therefore, the possibility of wrong recognition due to external light is decreased.
[0015] The plurality of infrared LEDs may each be located at any of positions arrayed in a matrix on a predetermined surface.
[0016] According to the above, the locations of the plurality of infrared LEDs are easily managed and grasped.
[0017] The predetermined surface may be located inclined with respect to an image capturing direction of the image capturing unit in a state where the predetermined surface is secured to the data transmission device.
[0018] According to the above, the size of the space for locating the light emitting unit is decreased.
[0019] The information processing apparatus may be a game apparatus main body. The data transmission device may be a game controller including an operation unit. The transmission section may transmit data including at least data from the operation unit of the data transmission device and the data generated based on the image captured by the image capturing unit.
[0020] According to the above, the game apparatus main body and the game controller are combined with the extended input device that does not have a function of communicating with the game apparatus main body to provide an information processing system.
[0021] The operation input portion may include a plurality of switches turned on or off in accordance with the user input and/or the use operation. The light emitting unit may include a plurality of light emitting members, and at least one of the plurality of light emitting members 203 corresponding to at least one of the switches turned on in accordance with the user input and/or the use operation may be made conductive to emit light.
[0022] According to the above, the plurality of light emitting members are caused to emit light by use of the plurality of switches in accordance with the user input and/or the user operation.
[0023] The operation input portions may include a plurality of contact members contactable with a body of the user. The light emitting unit may include a plurality of light emitting members, and at least one of the plurality of light emitting members corresponding to at least one of the plurality of contact members in contact with the body of the user may be made conductive to emit light.
[0024] According to the above, the plurality of light emitting members are caused to emit light by use of the plurality of contact members in accordance with the user input and/or the user operation.
[0025] The light emitting unit may cause different light emitting members, among the plurality of light emitting members, to emit light in accordance with an amount of the user input to, and/or the user operation on, the operation input member.
[0026] According to the above, the presence/absence of, and the amount of, the user input and/or the user operation is distinguished by the light emission of the light emitting members.
[0027] The data transmission device may further include a near field communication section. The near field communication section performs near field communication with another device. The extended input device may further include an information storage medium. The information storage medium is located at a position where the information storage medium is capable of performing the near field communication with the near field communication section in a state of being secured to the data transmission device by the securing portion, and performs the near field communication with the near field communication section. In this case, the transmission section may transmit data including at least data acquired from the information storage medium by the near field communication and the data generated based on the image captured by the image capturing unit. The processing section may execute the information process in accordance with the user input and/or the user operation made by use of the extended input device, based on the data acquired from the information storage medium and the data generated based on the image captured by the image capturing unit.
[0028] According to the above, a process based on the information stored on the information storage medium is made possible. For example, a process of determining the type of the extended input device, a process of authenticating the extended input device, or a process of acquiring calibration information on the extended input device is made possible.
[0029] The exemplary embodiment may be carried out in the form of an extended input device or an information processing method.
[0030] According to the exemplary embodiment, the extended input device is realized with a simple structure.
[0031] These and other objects, features, aspects and advantages of the exemplary embodiment will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS
[0138] For example, in the exemplary embodiment, as shown in FIG. 8, the extended input device 200 is a keyboard-type input device including a plurality of input keys 205. The user selects any one of the input keys 205 of the extended input device 200 and makes a touch operation or a press operation on the selected input key 205 with his/her finger. As a result, a letter corresponding to the operated input key 205 is input, a display object (e.g., a cursor) is moved in accordance with the operated input key 205, or a user instruction in accordance with the operated input key 205 is followed. Such a result of the input is displayed on the display device (e.g., the display 12 of the main body apparatus 2). A game in which a player object located in a virtual space makes a motion may be performed, or various types of information may be displayed, in accordance with an operation made by use of the extended input device 200. In the exemplary embodiment, a so-called keyboard may be used to make a user input to perform various processes.
Source: Search application number 15/687771 at http://appft.uspto.gov/netahtml/PTO/search-bool.html[0152] In this example, as shown in FIG. 14, the extended input device 250 is a drum-type controller including a plurality of operation surfaces 255a, 255b, 255x and 255y, an operation button 255s and a microphone 257m. The user beats at least one of the plurality of operation surfaces 255a, 255b, 255x and 255y, inputs a sound of a voice or the beat to the microphone 257m, or presses the operation button 255s. As a result, a player character PC makes a motion in accordance with the content of operation and displayed on the display device (e.g., the display 12 of the main body apparatus 2). In accordance with the operation made by use of the extended input device 250, a process of changing a virtual object located in the virtual space may be executed, the degree of matching of the operation timings may be determined, or various information may be displayed. Various game processes or information processes may be executed.
As always, a patent is not proof of a future product or service.
