diff --git "a/datasets/queries_content_with_features.json" "b/datasets/queries_content_with_features.json" new file mode 100644--- /dev/null +++ "b/datasets/queries_content_with_features.json" @@ -0,0 +1,7096 @@ +[ + { + "FAN": "103964109", + "CPC": [ + "G06F-003/0481/7", + "G06F-003/0482", + "G06F-003/0484/2", + "G06F-009/00", + "G06F-009/451", + "G06F-016/00", + "H04L-051/224" + ], + "Content": { + "title": "Notification message processing method and device", + "pa01": "A notification message processing method and a device are provided. The method includes: An electronic device obtains a target notification message of a target application, where the target notification message includes first rich media information, and the first rich media information includes one or more of voice information, video information, or an animated image ; the electronic device displays a first interface, where the first interface includes notification messages of at least two applications, the at least two applications include the target application, and the notification messages of the at least two applications include the target notification message ; and the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message . According to the method, rich media information such as a voice can be pushed to the user in a notification message of an APP, so that a form and content of the notification message are novel and vivid, and therefore are attractive to the user. Then, the user can play the rich media information in the notification message.", + "c-en-0001": "A notification message processing method, applied to an electronic device, wherein the method comprises: obtaining, by the electronic device, a target notification message of a target application, wherein the target notification message comprises first rich media information, and the first rich media information comprises one or more of voice information, video information, or an animated image; displaying, by the electronic device, a first interface, wherein the first interface comprises notification messages of at least two applications, the at least two applications comprise the target application, and the notification messages of the at least two applications comprise the target notification message; and playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message.", + "c-en-0002": "The method according to claim 1, wherein on the first interface, the target notification message comprises a prompt identifier, and the prompt identifier is for prompting that the target notification message comprises rich media information.", + "c-en-0003": "The method according to claim 2, wherein the playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message comprises: playing, by the electronic device, the first rich media information in the target notification message in response to an operation of tapping the prompt identifier in the target notification message by the user.", + "c-en-0004": "The method according to claim 1 or 2, wherein the target notification message comprises a notification title, and the playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message comprises: playing, by the electronic device, the first rich media information in the target notification message in response to an operation of tapping the notification title in the target notification message by the user.", + "c-en-0005": "The method according to claim 1 or 2, wherein the target notification message comprises a play control, and the playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message comprises: playing, by the electronic device, the first rich media information in the target notification message in response to an operation of tapping the play control in the target notification message by the user.", + "c-en-0006": "The method according to any one of claims 1 to 5, wherein the first rich media information in the target notification message is determined by a server of the target application based on a user feature corresponding to the electronic device, and the user feature comprises user behavior information or a user profile.", + "c-en-0007": "The method according to claim 6, wherein the first rich media information in the target notification message corresponds to a target media resource, the target media resource corresponds to a plurality of pieces of rich media information, and the first rich media information is one piece of rich media information determined, based on the user feature corresponding to the electronic device, by the server from the plurality of pieces of rich media information corresponding to the target media resource.", + "c-en-0008": "The method according to claim 7, wherein the target media resource is a media resource reserved by the user, the target media resource corresponds to rich media information of a plurality of associated persons, and the first rich media information is rich media information that is of a favorite associated person of the user corresponding to the electronic device and that is determined, based on the user feature corresponding to the electronic device, by the server from the plurality of pieces of rich media information corresponding to the target media resource.", + "c-en-0009": "The method according to any one of claims 1 to 8, wherein on the first interface, the target notification message comprises first prompt information, and the first prompt information is for prompting play duration of the first rich media information comprised in the target notification message.", + "c-en-0010": "The method according to any one of claims 1 to 9, wherein the method further comprises: displaying, by the electronic device, second prompt information, wherein the second prompt information is for prompting play progress of the first rich media information.", + "c-en-0011": "The method according to claim 7 or 8, wherein the target notification message further comprises a switching control, and the method further comprises: playing, by the electronic device in response to an operation performed by the user on the switching control in the target notification message, second rich media information corresponding to the target media resource.", + "c-en-0012": "The method according to claim 2 or 3, wherein the prompt identifier on the first interface is in a first state, and the playing, by the electronic device, the first rich media information in the target notification message comprises: displaying, by the electronic device, a second interface, and playing the first rich media information in the target notification message, wherein the second interface comprises the prompt identifier, the prompt identifier on the second interface is in a second state, and the second state is different from the first state.", + "c-en-0013": "The method according to claim 2 or 3, wherein the playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message comprises: displaying, by the electronic device, a third interface in response to the trigger operation performed by the user on the target notification message, wherein the third interface is an application interface of the target application; and playing, by the electronic device, the first rich media information in the target notification message by using the target application.", + "c-en-0014": "The method according to claim 13, wherein the target notification message comprises a notification title and prompt information of the target application, and the prompt information of the target application comprises an icon and/or a name of the target application; and the displaying, by the electronic device, a third interface in response to the trigger operation performed by the user on the target notification message, wherein the third interface is an application interface of the target application comprises: displaying, by the electronic device, the third interface in response to an operation of tapping the notification title or the prompt information of the target application in the target notification message by the user, wherein the third interface is the application interface of the target application.", + "c-en-0015": "The method according to claim 14, wherein the target notification message comprises the notification title, and the displaying, by the electronic device, a third interface in response to the trigger operation performed by the user on the target notification message, wherein the third interface is an application interface of the target application comprises: displaying, by the electronic device, the third interface in response to an operation of touching and holding the notification title in the target notification message by the user, wherein the third interface is the application interface of the target application.", + "c-en-0016": "The method according to any one of claims 1 to 11, wherein the playing, by the electronic device, the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message comprises: displaying, by the electronic device, a play mode list in response to the trigger operation performed by the user on the target notification message, wherein the play mode list comprises at least one application for playing the rich media information; and playing, by the electronic device in response to an operation of selecting an application from the play mode list by the user, the first rich media information in the target notification message by using the selected application.", + "c-en-0017": "The method according to any one of claims 1 to 16, wherein the electronic device stores a correspondence that is preset by the user and that is between an application and a rich media information type, the rich media information type comprises a voice information type, a video information type, or an animated image type, and a first rich media information type is consistent with a rich media information type corresponding to the target application in the correspondence.", + "c-en-0018": "The method according to any one of claims 1 to 17, wherein the target notification message that is of the target application and that is obtained by the electronic device comprises a resource identifier of the first rich media information, and the playing, by the electronic device, the first rich media information in the target notification message comprises: obtaining, by the electronic device, data content of the first rich media information based on the resource identifier of the first rich media information in the target notification message; and playing, by the electronic device, the first rich media information based on the data content of the first rich media information.", + "c-en-0019": "The method according to any one of claims 1 to 18, wherein the target notification message further comprises one or more of identification information of the target application, icon information of the target application, the notification title, a type of the notification message, or a device identifier corresponding to the electronic device.", + "c-en-0020": "A notification message processing method, applied to a server, wherein the method comprises: generating, by the server, a target notification message of a target application, wherein the target notification message comprises first rich media information, and the first rich media information comprises one or more of voice information, video information, or an animated image; and pushing, by the server, the target notification message to an electronic device.", + "c-en-0021": "The method according to claim 20, wherein the first rich media information in the target notification message corresponds to a target media resource, and the method further comprises: determining, by the server based on a user feature corresponding to the electronic device, the first rich media information from a plurality of pieces of rich media information corresponding to the target media resource.", + "c-en-0022": "An electronic device, comprising: a screen, configured to display an interface; one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, the one or more computer programs comprise instructions, and when the instructions are executed by the electronic device, the electronic device is enabled to perform the notification message processing method according to any one of claims 1 to 19, or perform the notification message processing method according to claim 20 or 21.", + "c-en-0023": "A computer-readable storage medium, comprising computer instructions, wherein when the computer instructions are run on a computer, the computer is enabled to perform the notification message processing method according to any one of claims 1 to 19, or perform the notification message processing method according to claim 20 or 21.", + "c-en-0024": "A computer program product, wherein when the computer program product runs on a computer, the computer is enabled to perform the notification message processing method according to any one of claims 1 to 19, or perform the notification message processing method according to claim 20 or 21.", + "p0001": "This application claims priority to Chinese Patent Application No. 202111007062.1, filed with the China National Intellectual Property Administration on August 30, 2021 and entitled \"NOTIFICATION MESSAGE PROCESSING METHOD AND DEVICE\", which is incorporated herein by reference in its entirety.", + "p0002": "TECHNICAL FIELD", + "p0003": "Embodiments of this application relate to the field of electronic technologies, and in particular, to a notification message processing method and a device.", + "p0004": "BACKGROUND", + "p0005": "With development of electronic technologies, more applications are used by users on electronic devices such as mobile phones. An APP server may push a notification message to an electronic device, and the electronic device may display the notification message to the user in a notification panel. If the user is interested in the notification message, the user may tap the notification message to start an APP and view the notification message.", + "p0006": "For example, for a schematic diagram of the notification message, refer to FIG. 1 . The notification message may usually include an application icon 11, an application name 12, a notification title 13, notification content 14, and the like. Currently, because the notification message is unattractive to the user, it is difficult to attract the user to open the notification message, and it is also difficult to attract, by pushing the notification message, the user to start the APP.", + "p0007": "SUMMARY", + "p0008": "Embodiments of this application provide a notification message processing method and a device, to push, to a user, rich media information such as a voice in a notification message pushed by an APP, so that a form and content of the notification message are novel and vivid, and therefore are attractive to the user. After playing the rich media information such as the voice in the notification message, the user is more likely to get interested and starts the APP. This can improve usage of the APP.", + "p0009": "To achieve the foregoing objectives, the following technical solutions are used in embodiments of this application.", + "p0010": "According to an aspect, an embodiment of this application provides a notification message processing method, applied to an electronic device. The method includes: The electronic device obtains a target notification message of a target application. The target notification message includes first rich media information, and the first rich media information includes one or more of voice information, video information, or an animated image. Then, the electronic device displays a first interface. The first interface includes notification messages of at least two applications, the at least two applications include the target application, and the notification messages of the at least two applications include the target notification message. The electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message.", + "p0011": "In this solution, the electronic device can push rich media information such as a voice to the user in the notification message of the target application, so that a form and content of the notification message are novel and vivid, and therefore are attractive to the user. After playing the rich media information such as the voice in the notification message, the user is more likely to get interested and starts the target application. This can improve usage of the target application.", + "p0012": "In a possible design, on the first interface, the target notification message includes a prompt identifier. The prompt identifier is for prompting that the notification message includes rich media information.", + "p0013": "In this way, the electronic device may intuitively remind, by using the prompt identifier, the user that the current notification message includes the rich media information.", + "p0014": "In another possible design, that the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message includes: The electronic device plays the first rich media information in the target notification message in response to an operation of tapping the prompt identifier in the target notification message by the user.", + "p0015": "In this way, the user may indicate, by triggering the prompt identifier, the electronic device to play the first rich media information in the notification message.", + "p0016": "In another possible design, the target notification message includes a notification title, and that the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message includes: The electronic device plays the first rich media information in the target notification message in response to an operation of tapping the notification title in the target notification message by the user.", + "p0017": "In this way, the user may indicate, by tapping the notification title in the notification message, the electronic device to play the first rich media information in the notification message.", + "p0018": "In another possible design, the target notification message includes a play control, and that the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message includes: The electronic device plays the first rich media information in the target notification message in response to an operation of tapping the play control in the target notification message by the user.", + "p0019": "In this way, the user may indicate, by tapping the play control in the notification message, the electronic device to play the first rich media information in the notification message.", + "p0020": "In another possible design, the first rich media information in the target notification message is determined by a server of the target application based on a user feature corresponding to the electronic device, and the user feature includes user behavior information or a user profile.", + "p0021": "In other words, the first rich media information is accurately determined based on the user feature. According to this solution, a personalized message can be pushed to the user.", + "p0022": "In another possible design, the first rich media information in the target notification message corresponds to a target media resource, the target media resource corresponds to a plurality of pieces of rich media information, and the first rich media information is one piece of rich media information determined, based on the user feature corresponding to the electronic device, by the server from the plurality of pieces of rich media information corresponding to the target media resource.", + "p0023": "In other words, in this solution, the first rich media information to be pushed to the user may be accurately determined, based on the user feature, from the plurality of pieces of rich media information corresponding to the target media resource, so that the personalized message is pushed to the user.", + "p0024": "In another possible design, the target media resource is a media resource reserved by the user, the target media resource corresponds to rich media information of a plurality of associated persons, and the first rich media information is rich media information that is of a favorite associated person of the user corresponding to the electronic device and that is determined, based on the user feature corresponding to the electronic device, from the plurality of pieces of rich media information corresponding to the target media resource.", + "p0025": "In other words, in this solution, the rich media information of the favorite associated person of the user may be accurately determined, based on the user feature, from the plurality of pieces of rich media information corresponding to the target media resource reserved by the user, so that the rich media information is pushed to the user in the notification message.", + "p0026": "In another possible design, on the first interface, the target notification message includes first prompt information, and the first prompt information is for prompting play duration of the first rich media information included in the notification message.", + "p0027": "That is, the electronic device may prompt the user with the play duration of the first rich media information on the interface.", + "p0028": "In another possible design, the method further includes: The electronic device displays second prompt information after starting to play the first rich media information in the target notification message, where the second prompt information is for prompting play progress of the first rich media information.", + "p0029": "In other words, in a process of playing the first rich media information, the electronic device may further prompt the user with the play progress in real time.", + "p0030": "In another possible design, the target notification message further includes a switching control, and the method further includes: The electronic device plays, in response to an operation performed by the user on the switching control in the target notification message, second rich media information corresponding to the target media resource.", + "p0031": "In other words, based on an indication operation of the user, the electronic device may switch between and play different rich media information included in the notification message.", + "p0032": "In another possible design, the prompt identifier on the first interface is in a first state, and that the electronic device plays the first rich media information in the target notification message includes: The electronic device displays a second interface, and plays the first rich media information in the target notification message. The second interface also includes the prompt identifier, and the prompt identifier on the second interface is in a second state. The second state is different from the first state.", + "p0033": "In this solution, a display status of the prompt identifier in the process of playing the first rich media information is different from a display status that is of the prompt identifier and that corresponds to a case in which the first rich media information is not played. The user may intuitively determine, based on the display status of the prompt identifier, whether the first rich media information is in a play state.", + "p0034": "In another possible design, that the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message includes: The electronic device displays a third interface in response to the trigger operation performed by the user on the target notification message, where the third interface is an application interface of the target application. The electronic device plays the first rich media information in the target notification message by using the target application.", + "p0035": "In this solution, the electronic device may play, based on the indication operation of the user, the rich media information in the notification message by using the target application for pushing the notification message.", + "p0036": "In another possible design, the target notification message includes a notification title and prompt information of the target application, and the prompt information of the target application includes an icon and/or a name of the target application. That the electronic device displays a third interface in response to the trigger operation performed by the user on the target notification message, where the third interface is an application interface of the target application includes: The electronic device displays the third interface in response to an operation of tapping the notification title or the prompt information of the target application in the target notification message by the user, where the third interface is the application interface of the target application.", + "p0037": "In this solution, the user may indicate, by tapping the notification title or the prompt information of the target application in the notification message, the electronic device to play the rich media information in the notification message by using the target application.", + "p0038": "In another possible design, the target notification message includes the notification title, and that the electronic device displays a third interface in response to the trigger operation performed by the user on the target notification message, where the third interface is an application interface of the target application includes: The electronic device displays the third interface in response to an operation of touching and holding the notification title in the target notification message by the user, where the third interface is the application interface of the target application.", + "p0039": "In this solution, the user may indicate, by touching and holding the notification title in the notification message, the electronic device to play the rich media information in the notification message by using the target application.", + "p0040": "In another possible design, that the electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by a user on the target notification message includes: The electronic device displays a play mode list in response to the trigger operation performed by the user on the target notification message, where the play mode list includes at least one application for playing the rich media information. The electronic device plays, in response to an operation of selecting an application from the play mode list by the user, the first rich media information in the target notification message by using the selected application.", + "p0041": "In this solution, the user may select, from the play mode list, the application for playing the rich media information, and the electronic device plays the first rich media information by using the application selected by the user. In this way, a play mode of the first rich media information is flexible.", + "p0042": "In another possible design, the first interface further includes another switching control, and the method further includes: The electronic device displays a play mode list in response to an operation performed by the user on the another switching control in the target notification message, where the play mode list includes one or more of a play mode corresponding to an operating system basic capability, the target application, and a second application. In response to an operation of selecting a target play mode from the play mode list by the user, the electronic device plays the first rich media information in the target notification message in the target play mode.", + "p0043": "In this solution, the user may trigger the switching control, so that the electronic device displays the play mode list. Then, the user may select, from the play mode list, the application for playing the rich media information, and the electronic device plays the first rich media information by using the application selected by the user. In this way, a play mode of the first rich media information is flexible.", + "p0044": "In another possible design, the electronic device stores a correspondence that is preset by the user and that is between an application and a rich media information type. The rich media information type includes a voice information type, a video information type, or an animated image type, and a first rich media information type is consistent with a rich media information type corresponding to the target application in the correspondence.", + "p0045": "In this way, the user may preset the correspondence between the application and the rich media type in the notification message, and the electronic device may display, to the user based on setting of the user for the application, a notification message including the rich media information type corresponding to the application.", + "p0046": "In another possible design, the target notification message that is of the target application and that is obtained by the electronic device includes a resource identifier of the first rich media information, and that the electronic device plays the first rich media information in the target notification message includes: The electronic device obtains data content of the first rich media information based on the resource identifier of the first rich media information in the target notification message. The electronic device plays the first rich media information based on the data content of the first rich media information.", + "p0047": "For example, the resource identifier may be a resource link. The electronic device may download the data content of the first rich media information from a network based on the resource link, to perform playing.", + "p0048": "In another possible design, the target notification message further includes one or more of identification information of the target application, icon information of the target application, the notification title, a type of the notification message, or a device identifier corresponding to the electronic device.", + "p0049": "In this way, the electronic device may obtain, from the notification message, information such as the identification information of the target application, the icon information of the target application, the notification title, the type of the notification message, or the device identifier corresponding to the electronic device.", + "p0050": "According to another aspect, an embodiment of this application provides a notification message processing method, applied to a server. The method includes: The server generates a target notification message of a target application, where the target notification message includes first rich media information, and the first rich media information includes one or more of voice information, video information, or an animated image. The server pushes the target notification message to an electronic device.", + "p0051": "In this solution, the server can push rich media information such as a voice to the electronic device in the notification message of the target application, so that a form and content of the notification message are novel and vivid, and therefore are attractive to a user. After playing the rich media information such as the voice in the notification message, the user is more likely to get interested and starts the target application. This can improve usage of the target application.", + "p0052": "In a possible design, the first rich media information in the target notification message corresponds to a target media resource, and the method further includes: The server determines, based on a user feature corresponding to the electronic device, the first rich media information from a plurality of pieces of rich media information corresponding to the target media resource.", + "p0053": "In this solution, the server may accurately determine, based on the user feature and from the plurality of pieces of rich media information corresponding to the target media resource, the first rich media information to be pushed to the user, to push a personalized message to the user.", + "p0054": "In another possible design, the target media resource is a media resource reserved by the user, and the target media resource corresponds to rich media information of a plurality of associated persons. The server determines, based on the user feature corresponding to the electronic device and from the plurality of pieces of rich media information corresponding to the target media resource, the first rich media information of a favorite associated person of the user corresponding to the electronic device .", + "p0055": "In this solution, the server may accurately determine, based on the user feature and from the plurality of pieces of rich media information corresponding to the target media resource reserved by the user, the first rich media information that is of the favorite associated person of the user and that is to be pushed to the user, to push the personalized message to the user.", + "p0056": "According to another aspect, an embodiment of this application provides a message processing apparatus. The apparatus is included in an electronic device, and the apparatus has a function of implementing behavior of the electronic device in any method according to the foregoing aspects and the possible designs of the foregoing aspects. The function may be implemented by hardware, or may be implemented by hardware executing corresponding software. The hardware or the software includes at least one module or unit corresponding to the function, for example, an obtaining module or unit, a display module or unit, and a playing module or unit.", + "p0057": "According to another aspect, an embodiment of this application provides an electronic device. The electronic device may include a screen, configured to display an interface; one or more processors; a memory; and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the electronic device is enabled to perform the notification message processing method according to any one of the possible designs of the foregoing aspects.", + "p0058": "According to another aspect, an embodiment of this application provides an electronic device. The electronic device may include one or more processors, a memory, and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the electronic device is enabled to perform the notification message processing method according to any one of the possible designs of the foregoing aspects.", + "p0059": "According to another aspect, an embodiment of this application provides a computer-readable storage medium, including computer instructions. When the computer instructions are run on an electronic device, the electronic device is enabled to perform the notification message processing method according to any one of the possible designs of the foregoing aspects.", + "p0060": "According to another aspect, an embodiment of this application provides a computer program product. When the computer program product runs on a computer, the computer is enabled to perform the notification message processing method performed by the electronic device according to any one of the possible designs of the foregoing aspects.", + "p0061": "According to another aspect, an embodiment of this application provides a server. The server may include one or more processors, a memory, and one or more computer programs. The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the processor, the server is enabled to perform the notification message processing method according to any one of the possible designs of the foregoing aspects.", + "p0062": "According to another aspect, an embodiment of this application provides a computer-readable storage medium, including computer instructions. When the computer instructions are run on a server, the server is enabled to perform the notification message processing method according to any one of the possible designs of the foregoing aspects.", + "p0063": "According to another aspect, an embodiment of this application provides a computer program product. When the computer program product runs on a computer, the computer is enabled to perform the notification message processing method performed by the server according to any one of the possible designs of the foregoing aspects.", + "p0064": "For beneficial effects of other aspects in this application, refer to related descriptions of beneficial effects in the foregoing method aspects. Details are not described again.", + "p0065": "BRIEF DESCRIPTION OF DRAWINGS", + "p0066": "FIG. 1 is a schematic diagram of a display effect of a notification message according to the conventional technology; FIG. 2 is a schematic diagram of an architecture of a message push system according to an embodiment of this application; FIG. 3 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of this application; FIG. 4 is a schematic diagram of a software module of an electronic device according to an embodiment of this application; FIG. 5A is a schematic diagram of a notification message push process according to an embodiment of this application; FIG. 5B is a schematic diagram of another notification message push process according to an embodiment of this application; FIG. 6 is a schematic diagram of another notification message push process according to an embodiment of this application; FIG. 7 is a schematic diagram of a correspondence between voice information in a notification message and a user according to an embodiment of this application; FIG. 8 is a schematic diagram of a structure of a notification message according to an embodiment of this application; FIG. 9 to FIG. 9 are a schematic diagram of a group of interfaces according to an embodiment of this application; FIG. 10 and FIG. 10 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 11 to FIG. 11 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 12 to FIG. 12 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 13 and FIG. 13 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 14 and FIG. 14 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 15 and FIG. 15 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 16 to FIG. 16 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 17 and FIG. 17 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 18 to FIG. 18 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 19 and FIG. 19 are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 20A to FIG. 20A are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 20B and FIG. 20B are a schematic diagram of another group of interfaces according to an embodiment of this application; FIG. 21 is a schematic diagram of an interface according to an embodiment of this application; FIG. 22 is a flowchart of processing a notification message according to an embodiment of this application; and FIG. 23 is a schematic diagram of a hardware structure of another electronic device according to an embodiment of this application.", + "p0067": "DESCRIPTION OF EMBODIMENTS", + "p0068": "The following describes the technical solutions in embodiments of this application with reference to the accompanying drawings in embodiments of this application. In description in embodiments of this application, \"/\" means \"or\" unless otherwise specified. For example, A/B may indicate A or B. In this specification, \"and/or\" describes only an association relationship for describing associated objects and indicates that three relationships may exist. For example, A and/or B may indicate the following three cases: Only A exists, both A and B exist, and only B exists. In addition, in the descriptions in embodiments of this application, \"a plurality of\" means two or more.", + "p0069": "The terms \"first\" and \"second\" mentioned below are merely intended for a purpose of description, and shall not be understood as an indication or implication of relative importance or implicit indication of a quantity of indicated technical features. Therefore, a feature limited by \"first\" or \"second\" may explicitly or implicitly include one or more features. In the descriptions of embodiments, unless otherwise specified, \"a plurality of\" means two or more.", + "p0070": "In embodiments of this application, the word \"example\", \"for example\", or the like is used to indicate giving an example, an illustration, or a description. Any embodiment or design scheme described as \"example\" or \"for example\" in embodiments of this application should not be explained as being more preferred or having more advantages than another embodiment or design scheme. Exactly, use of the word \"example\", \"for example\", or the like is intended to present a relative concept in a specific manner.", + "p0071": "Currently, in a notification message pushed by an APP server and displayed by an electronic device, as shown in FIG. 1 , notification content is usually displayed in a form of text description, and is unattractive to a user. In addition, limited by a display range of a screen, text content in the notification message is usually concise and refined, has a small quantity of words, and has a small amount of information. As a result, it is difficult to arouse interest of the user. In addition, as shown in FIG. 1 , a plurality of different notification messages corresponding to a same APP or different APPs all are displayed in a form of a plain text, and display effects are similar . As a result, it is difficult for the user to find, from the plurality of notification messages, a notification message that the user is interested in, and most users directly ignore these notification messages.", + "p0072": "Embodiments of this application provide a notification message processing method, to push, to a user, rich media information such as a voice in a notification message pushed by an APP server, so that a form and content of the notification message are novel and vivid, and therefore are attractive to the user. After playing the rich media information such as the voice in the notification message, the user is more likely to get interested and starts an application APP . In this way, usage of the APP is high. The rich media information includes information such as an animation, a sound, and a video, or any combination thereof.", + "p0073": "The notification message processing method provided in embodiments of this application may be applied to a message push system. For example, as shown in FIG. 2 , the message push system 10 may include an electronic device 100, at least one APP server 200, a push server 300, and the like. The electronic device may include a plurality of APPs that may provide experience of a plurality of different functions for a user. For example, the APPs included in the electronic device may be a video playing APP, an audio playing APP, a music playing APP, a news APP, a game APP, an e-book APP, a financial APP, or the like. One APP may correspond to one APP server. One APP server may correspond to one or more APPs and provide corresponding service support for a corresponding APP function. The at least one APP server in the push system 10 may include a server corresponding to the video playing APP, a server corresponding to the news APP, a server corresponding to the audio playing APP, and the like. The APP server is configured to: push a notification message of a corresponding APP, and store and process related data for a provided APP function. The push server is configured to forward, to the electronic device, the notification message pushed by the APP server, and may further have functions such as message link management and message construction.", + "p0074": "To be specific, in the message push system, as shown in FIG. 2 , the APP server may push the notification message to the push server, the push server sends the notification message to the electronic device, and the electronic device displays the notification message on a screen, to display the notification message to the user. The notification message pushed by the APP server may also be understood as a notification message corresponding to an APP, a notification message from an APP, a notification message of an APP, or the like.", + "p0075": "The notification message pushed by the APP server is automatically pushed by the APP server. However, in an advertisement, a notification message can be obtained only after being requested from the server. In addition, when the electronic device does not start an APP, a notification message of the APP may also be displayed on the screen of the electronic device. For example, when an APP 1 is not started, the electronic device displays a notification panel in response to a pull-down operation performed by the user on the top of the electronic device, and displays a notification message of the APP 1 in the notification panel. It may be understood that the electronic device may display notification messages of one or more APPs on the screen , and there are also a plurality of notification messages for a same APP.", + "p0076": "In addition, the user can disable the notification message of the APP in the APP through setting. In this case, the electronic device no longer displays the notification message of the APP to the user on the screen, and the electronic device may further receive the notification message pushed by the APP server, or may no longer receive the notification message pushed by the APP server. Subsequently, after the user enables the notification message of the APP in the APP through setting, the electronic device may re-display the notification message of the APP to the user on the screen.", + "p0077": "In an embodiment of this application, there may be a plurality of policies for pushing the notification message by the APP server. For example, the APP server may periodically push the notification message. For example, the APP server may push a notification message at 8:00 AM every day, or the APP server pushes a notification message when a media resource reserved by the user is about to be available. For another example, the APP server may push the notification message in response to event triggering. For example, after the user purchases a membership or a voucher, or executes another event by using the APP, the APP may be triggered to push a notification message. For another example, the APP server may push a notification message in response to active triggering of an APP operation person. For example, for the news APP, when there is major news, the APP operation person may actively trigger the APP server to push a notification message. A specific policy for pushing the notification message by the APP server is not limited in embodiments of this application.", + "p0078": "The electronic device in the message push system shown in FIG. 2 may be a mobile phone, a tablet computer, a wearable device, a smart home device, a vehicle-mounted device, or an augmented reality /virtual reality device, a notebook computer, an ultra-mobile personal computer , a netbook, a personal digital assistant , or another device. A device type of the electronic device is not limited in embodiments of this application.", + "p0079": "For example, FIG. 3 is a schematic diagram of a structure of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, a subscriber identity module card interface 195, and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, an optical proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.", + "p0080": "The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor , a modem processor, a graphics processing unit , an image signal processor , a controller, a memory, a video codec, a digital signal processor , a baseband processor, a neural-network processing unit , and/or the like. Different processing units may be independent components, or may be integrated into one or more processors.", + "p0081": "The controller may be a nerve center and a command center of the electronic device 100. The controller may generate an operation control signal based on instruction operation code and a time sequence signal, to complete control of instruction reading and instruction execution.", + "p0082": "A memory may be further disposed in the processor 110, and is configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data just used or cyclically used by the processor 110. If the processor 110 needs to use the instructions or the data again, the processor may directly invoke the instructions or the data from the memory. This avoids repeated access and reduces waiting time of the processor 110, to improve system efficiency.", + "p0083": "A wireless communication function of the electronic device 100 may be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, the baseband processor, and the like.", + "p0084": "The antenna 1 and the antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 may be configured to cover one or more communication frequency bands. Different antennas may be further multiplexed, to improve antenna utilization. For example, the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antenna may be used in combination with a tuning switch.", + "p0085": "The mobile communication module 150 can provide a solution, applied to the electronic device 100, to wireless communication including 2G/3G/4G/5G, or the like. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier , and the like. The mobile communication module 150 may receive an electromagnetic wave through the antenna 1, perform processing such as filtering or amplification on the received electromagnetic wave, and transmit a processed electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may further amplify a signal modulated by the modem processor, and convert an amplified signal into an electromagnetic wave for radiation through the antenna 1. In some embodiments, at least some functional modules in the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some functional modules in the mobile communication module 150 and at least some modules in the processor 110 may be disposed in a same device.", + "p0086": "The modem processor may include a modulator and a demodulator. The modulator is configured to modulate a to-be-sent low-frequency baseband signal into a medium/high-frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. Then, the demodulator transmits the low-frequency baseband signal obtained through demodulation to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then transmitted to the application processor. The application processor outputs a sound signal through an audio device , or displays an image or a video through the display 194. In some embodiments, the modem processor may be an independent component. In some other embodiments, the modem processor may be independent of the processor 110, and is disposed in the same device as the mobile communication module 150 or another functional module.", + "p0087": "The wireless communication module 160 may provide a solution, applied to the electronic device 100, to wireless communication including a wireless local area network network), Bluetooth , a global navigation satellite system , frequency modulation , a near field communication technology, an infrared technology, or the like. The wireless communication module 160 may be one or more components integrating at least one communication processing module. The wireless communication module 160 receives an electromagnetic wave through the antenna 2, performs frequency modulation and filtering processing on an electromagnetic wave signal, and sends a processed signal to the processor 110. The wireless communication module 160 may further receive a to-be-sent signal from the processor 110, perform frequency modulation and amplification on the signal, and convert a processed signal into an electromagnetic wave for radiation through the antenna 2.", + "p0088": "In some embodiments, in the electronic device 100, the antenna 1 and the mobile communication module 150 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and another device by using a wireless communication technology. The wireless communication technology may include a global system for mobile communications , a general packet radio service , code division multiple access , wideband code division multiple access , time-division code division multiple access , long term evolution , BT, a GNSS, a WLAN, NFC, FM, an IR technology, and/or the like. The GNSS may include a global positioning system , a global navigation satellite system , a BeiDou navigation satellite system , a quasi-zenith satellite system , and/or a satellite based augmentation system", + "p0089": "In embodiments of this application, the mobile communication module 150 or the wireless communication module 160 may be configured to receive a notification message pushed by an APP server through the push server.", + "p0090": "The electronic device 100 may implement a display function through the GPU, the display 194, the application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display 194 and the application processor. The GPU is configured to: perform mathematical and geometric calculation, and render an image. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.", + "p0091": "The display 194 is configured to display an image, a video, and the like. The display 194 includes a display panel. The display panel may be a liquid crystal display , an organic light-emitting diode , an active-matrix organic light emitting diode , a flexible light-emitting diode , a mini-LED, a micro-LED, a micro-OLED, a quantum dot light emitting diode , or the like. In some embodiments, the electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1. In this embodiment of this application, the display 194 may be configured to display, in the notification panel or in another location like at the top of the display 194, the notification message pushed by the APP server. Notification content of the notification message may include rich media information such as a voice. When the notification content of the notification message includes an animated image or a video , the display 194 may be further configured to display the animated image or video content in the notification message.", + "p0092": "The electronic device 100 may implement a photographing function through the ISP, the camera 193, the video codec, the GPU, the display 194, the application processor, and the like.", + "p0093": "The ISP is configured to process data fed back by the camera 193. For example, during photographing, a shutter is pressed, and light is transmitted to a photosensitive element of the camera through a lens. An optical signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, to convert the electrical signal into a visible image. The ISP may further perform algorithm optimization on noise, brightness, and complexion of the image. The ISP may further optimize parameters such as exposure and a color temperature of a photographing scenario. In some embodiments, the ISP may be disposed in the camera 193.", + "p0094": "The camera 193 is configured to capture a static image or a video. An optical image of an object is generated through the lens, and is projected onto the photosensitive element. The photosensitive element may be a charge-coupled device or a complementary metal-oxide-semiconductor phototransistor. The photosensitive element converts an optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert the electrical signal into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format like RGB or YUV. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.", + "p0095": "The digital signal processor is configured to process a digital signal, and may process another digital signal in addition to the digital image signal. For example, when the electronic device 100 selects a frequency, the digital signal processor is configured to perform Fourier transform on frequency energy.", + "p0096": "The video codec is configured to compress or decompress a digital video. The electronic device 100 may support one or more types of video codecs. In this way, the electronic device 100 may play or record videos in a plurality of coding formats, for example, moving picture experts group -1, MPEG-2, MPEG-3, and MPEG-4.", + "p0097": "The external memory interface 120 may be configured to connect to an external memory card, for example, a micro SD card, to extend a storage capability of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120, to implement a data storage function. For example, files such as music and a video are stored in the external memory card.", + "p0098": "The internal memory 121 may be configured to store computer-executable program code. The executable program code includes instructions. The processor 110 runs the instructions stored in the internal memory 121, to perform various function applications of the electronic device 100 and data processing. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required by at least one function , and the like. The data storage area may store data created during use of the electronic device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a non-volatile memory, for example, at least one magnetic disk storage device, a flash memory, or a universal flash storage .", + "p0099": "The electronic device 100 may implement an audio function, for example, music playing or recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headset jack 170D, the application processor, and the like.", + "p0100": "The audio module 170 is configured to convert digital audio information into an analog audio signal output, and is also configured to convert an analog audio input into a digital audio signal. The audio module 170 may be further configured to encode and decode an audio signal. In some embodiments, the audio module 170 may be disposed in the processor 110, or some functional modules in the audio module 170 are disposed in the processor 110. In embodiments of this application, when the notification content of the notification message includes rich media information such as a voice or a video, the audio module 170 may be further configured to play the voice or video content in the notification message.", + "p0101": "The speaker 170A, also referred to as a \"loudspeaker\", is configured to convert an audio electrical signal into a sound signal. The electronic device 100 may be configured to listen to music or answer a call in a hands-free mode through the speaker 170A.", + "p0102": "The receiver 170B, also referred to as an \"earpiece\", is configured to convert an audio electrical signal into a sound signal. When a call is answered or voice information is received through the electronic device 100, the receiver 170B may be put close to a human ear to listen to a voice.", + "p0103": "The microphone 170C, also referred to as a \"mike\" or a \"mic\", is configured to convert a sound signal into an electrical signal.", + "p0104": "The headset interface 170D is configured to connect to a wired headset, and may specifically include a plurality of different interfaces .", + "p0105": "The touch sensor 180K is also referred to as a \"touch panel\". The touch sensor 180K may be disposed on the display 194, and the touch sensor 180K and the display 194 constitute a touchscreen, which is also referred to as a \"touch screen\". The touch sensor 180K is configured to detect a touch operation performed on or near the touch sensor. The touch sensor may transfer the detected touch operation to the application processor, to determine a type of a touch event. The display 194 may provide a visual output related to the touch operation. In some other embodiments, the touch sensor 180K may alternatively be disposed on a surface of the electronic device 100 at a location different from that of the display 194.", + "p0106": "It may be understood that the structure shown in this embodiment of this application does not constitute a specific limitation on the electronic device 100. In some other embodiments of this application, the electronic device 100 may include more or fewer components than those shown in the figure, or some components may be combined, or some components may be split, or different component arrangements may be used. The components shown in the figure may be implemented by using hardware, software, or a combination of software and hardware.", + "p0107": "In embodiments of this application, the mobile communication module 150 or the wireless communication module 160 may be configured to receive the notification message pushed by the APP server through the push server. The display 194 may be configured to display, in the notification panel or in another location, the notification message pushed by the APP server. The notification content of the notification message may include rich media information such as a voice. In this way, a form and content of the notification message are novel and vivid, and therefore are attractive to a user. When the notification content of the notification message includes an animated image or a video, the display 194 may be further configured to display the animated image or video content in the notification message. When the notification content of the notification message includes rich media information such as a voice or a video, the audio module 170 may be further configured to play the voice or video content in the notification message.", + "p0108": "From a perspective of a software module, as shown in FIG. 4 , the electronic device may include various APPs and a plurality of modules such as an operating system basic service module. The operating system basic service module may include a notification processing module for notification processing, a push agent module, or the like). The notification processing module may be configured to: receive a notification message pushed by the APP server through the push server, parse the notification message, and display the notification message in the notification panel or in another location like at the top of a screen of the electronic device. The operating system basic service module may further include a voice playing module , a video playing module, or some other basic service modules. The voice playing module in the operating system basic service module may be configured to play voice information in the notification message by using an operating system basic capability of the electronic device, so that the user can quickly hear the voice information without starting a specific APP to perform voice playing. The video playing module in the operating system basic service module may be configured to play video information in the notification message by using the operating system basic capability of the electronic device, so that the user can quickly view the video information without starting a specific APP to perform video playing.", + "p0109": "In this way, in the message push system, the APP server may push the notification message to the push server, the push server sends the notification message to the notification processing module in the electronic device, and the electronic device displays, to the user, the notification message obtained by the notification processing module.", + "p0110": "When the user does not log in to an APP, as shown in FIG. 5A , a message push process in an embodiment of this application may include: When the user installs the APP on the electronic device and starts the APP, the APP on the electronic device requests, from the notification processing module, to obtain a device identifier , a media access control address, an open advertising identifier , or the like) of the electronic device. The notification processing module transfers the device identifier to the APP, and the APP sends the device identifier to the APP server. The APP server records the device identifier of the electronic device. Subsequently, when the APP server needs to push the notification message, the APP server sends the notification message and the recorded device identifier of the electronic device to the push server. The push server sends the notification message to the electronic device corresponding to the device identifier. After obtaining the notification message, the notification processing module in the electronic device displays the notification message to the user in the notification panel or in another location displayed on the screen.", + "p0111": "In some other embodiments, the message push system may further include an authentication server, configured to perform authentication and verification on information such as a user name, a password, and a login token after the user logs in to the APP.", + "p0112": "For example, when the user logs in to the APP, as shown in FIG. 5B , a message push process in an embodiment of this application may include: The user starts the APP installed on the electronic device, and inputs a user name and a password to request login, and then the APP sends the user name and the password to the authentication server. After verification on the user name and the password succeeds, the authentication server sends a user login token to the APP on the electronic device. The APP requests, from the notification processing module, to obtain the device identifier of the electronic device. The notification processing module transfers the device identifier to the APP, and the APP sends the device identifier and the login token to the APP server. The APP server requests, from the authentication server, to verify validity of the login token. If verification on the validity of the login token succeeds, the authentication server sends a user identifier to the APP server, and the APP server records a correspondence between the user identifier and the device identifier. Subsequently, when the APP server needs to push a notification message to a user, the APP server may send, to the push server, the notification message and the device identifier corresponding to the user identifier of the user. The push server sends the notification message to the electronic device corresponding to the device identifier. The notification processing module in the electronic device receives the notification message, and displays the notification message to the user in the notification panel or in another location displayed on the screen.", + "p0113": "The notification message processing method provided in embodiments of this application is described below by using an example in which the electronic device is a mobile phone having the structures shown in FIG. 3 and FIG. 4 . The method includes the following steps.", + "p0114": "An APP 1 is installed and started on the mobile phone. The APP 1 may be a system application, or may be a third-party application. The APP 1 may be a video playing application , an audio playing application, a music playing application, a news application, a game application, an e-book application, a financial application, or the like. A specific type of the APP 1 is not limited in embodiments of this application. Then, an APP 1 server pushes a notification message to the mobile phone on which the APP 1 is installed and started.", + "p0115": "As described above, the APP 1 server may push the notification message to the mobile phone periodically, or in response to event triggering, or in response to triggering of an APP 1 operation person. As described above, when a user logs in to or does not log in to the APP 1, the mobile phone may receive the notification message pushed by the APP 1 server. In addition, as described above, the notification message may include rich media information such as a voice, an animation, or a video.", + "p0116": "In addition, the APP 1 server may further accurately push the notification message to the user, to push notification content that the user is interested in as much as possible. For example, the APP 1 server may determine a preference of the user based on user behavior information, to accurately push the notification message to the user. For another example, the APP 1 server may accurately push the notification message to the user based on a user profile. The user profile may indicate a preference or habit of the user, or the like. The APP 1 server may accurately push the notification message to the user based on the preference or habit of the user, to push the notification content that the user is interested in as much as possible. The user profile may be obtained by the APP 1 server by analyzing behavior information of the user for the APP 1, or may be obtained by the mobile phone or a cloud server by analyzing behavior information of the user using the mobile phone, or may be obtained by the mobile phone by analyzing identity information, preference information, or other information input by the user.", + "p0117": "An example in which the APP 1 is a video playing application, the APP 1 is logged in on the mobile phone, the APP 1 server pushes a notification message to the mobile phone used by the user when a movie reserved by the user is about to be available, and the notification message includes voice information is used for description. Refer to a procedure shown in a dashed box in FIG. 5B that corresponds to a case in which the user logs in to the APP 1. The APP 1 server records the correspondence between the user identifier and the device identifier, including a correspondence between a user identifier 1 corresponding to a user 1 and a device identifier 1 of a mobile phone 1. For the notification message processing method implemented after the procedure shown in the dashed box in FIG. 5B , refer to a procedure shown in FIG. 6 .", + "p0118": "As shown in FIG. 6 , the user 1 reserves a movie 1 on the APP 1, and the APP 1 server records a correspondence between the user identifier 1 corresponding to the user 1 and an identifier of the reserved movie 1. In this way, when the movie 1 is about to be available, a notification message related to the movie 1 is pushed to the mobile phone corresponding to the user identifier 1, and then the notification message related to the movie 1 is pushed to the user 1.", + "p0119": "Still as shown in FIG. 6 , when determining that the movie 1 is about to be available , the APP 1 server determines, by querying the correspondence between the movie 1 and the user identifier, a user identifier list for which the movie 1 is reserved. The APP 1 server determines that the user identifier list includes the user identifier 1, and the user corresponding to the user identifier 1 reserves the movie 1. The APP 1 server determines, based on the previously recorded correspondence between the user identifier and the device identifier, a device identifier list corresponding to the user identifier list for which the movie 1 is reserved. The device identifier list includes the device identifier 1 corresponding to the user identifier 1.", + "p0120": "As shown in FIG. 6 , the APP 1 server accurately determines target voice information to be pushed to the user 1 corresponding to the user identifier 1, where the target voice information meets a preference of the user 1 corresponding to the user identifier 1. The target voice information is pushed to the user 1 by using the notification message, so that the notification message that the user 1 is interested in can be pushed as accurately as possible.", + "p0121": "The voice information in the notification message that may be pushed by the APP 1 server is a voice related to the movie 1, for example, a publicity voice of the movie 1. For example, the voice information may be a publicity voice recorded by a related filmmaker like an actor, a director, or a producer of the movie 1, or may be an opening song, an ending song, or an episode of the movie. For example, for the voice related to the movie 1, refer to in FIG. 7 . The voice information in the notification message may be specifically pushed in a plurality of manners. For example, the notification message carries a resource identifier of the voice information for push. For another example, the notification message carries data content of the voice information for push.", + "p0122": "The APP 1 server may accurately determine, based on a user feature of the user 1 corresponding to the user identifier 1, the target voice information in the notification message to be pushed to the user 1. The user feature may include information such as user behavior information or a user profile. For example, the APP 1 server may perform analysis to determine a person that the user 1 likes based on a behavior of liking a specific person, a behavior of watching clips including only a specific person, a behavior of following a specific person, a behavior of searching for a specific person, a behavior of making a comment on a specific person, or the like that is conducted by the user 1 corresponding to the user identifier 1 in a process of using the APP 1. Then, the APP 1 server determines that the target voice information in the to-be-pushed message is a voice of the person that the user 1 likes. For example, the user 1 follows an actor 1 in the APP 1 on the mobile phone 1, or boosts popularity for the actor 1 in the APP 1, or watches a large quantity of works of the actor 1 in the APP 1, or searches for the actor 1 in the APP 1, or searches for the actor 1 a large quantity of times in the APP 1, or chooses to watch clips including only the actor 1 in a video in the APP 1, or makes comments on the actor 1 in the APP 1 a large quantity of times, or the like. The APP 1 server analyzes the user behavior information of the user 1 corresponding to the user identifier 1, to determine that the user 1 likes the actor 1. In this way, the APP 1 server may push, in the notification message, a voice of the actor 1 in the movie 1 to the user 1, for example, a publicity voice recorded by the actor 1. In other words, in the to-be-pushed notification message determined by the APP 1 server, the target voice information is the publicity voice recorded by the actor 1.", + "p0123": "If the APP 1 server determines, based on the user feature, that the user 1 corresponding to the user identifier 1 likes a plurality of persons related to the movie 1, in an implementation, the APP 1 server performs priority sorting based on a degree of preference of the user 1 for actors, selects a favorite person of the user in FIG. 7 , the favorite person of the user 1 is the actor 1), and uses a voice of the person as the target voice information in the to-be-pushed notification message. In another implementation, the APP 1 server uses, as the target voice information in the to-be-pushed notification message, voices of the plurality of persons in FIG. 7 , favorite persons of the user 1 include the actor 1 and an actor 3) that the user 1 likes. The APP 1 server may separately use, as the target voice information, a voice of each person that the user 1 likes, and then separately push the target voice information to the user 1 by using one notification message; or may use, as the target voice information, the voices of the plurality of persons that the user 1 likes, and then push, by using one notification message, the target voice information to the mobile phone 1 used by the user 1, to push the target voice information to the user 1.", + "p0124": "If the APP 1 server cannot determine a person that is associated with the movie 1 and that the user 1 corresponding to the user identifier 1 likes, the APP 1 server may use, as the target voice information, first N pieces of voice information that are arranged in a preset sequence and that are in voice information corresponding to the movie 1, to push the target voice information to the mobile phone 1 and the user 1. For example, the voice information corresponding to the movie 1 may be arranged in a sequence of a director, a leading actor, and a producer. For another example, the voice information corresponding to the movie 1 may be arranged in descending order from the most popular person to the least popular person determined based on statistics collected by the APP 1 server. For another example, the voice information corresponding to the movie 1 may be arranged in a sequence of name initials of persons.", + "p0125": "Still as shown in FIG. 6 , after determining the target voice information in the notification message to be pushed to the user 1 corresponding to the user identifier 1, the APP 1 server sends, to the push server, the notification message including the target voice information and the device identifier 1 corresponding to the user identifier 1. In some embodiments, the notification message includes a resource identifier of the target voice information. For example, the resource identifier may be a uniform resource identifier , a uniform resource locator , or the like. A specific type of the resource identifier is not limited in embodiments of this application. In some other embodiments, the notification message includes data content of the target voice information.", + "p0126": "It may be understood that, in addition to the user 1 corresponding to the user identifier 1, another user may also reserve the movie 1, and the APP 1 may accurately push the notification message to the another user based on user behavior information or a user profile corresponding to the another user identifier. For example, as shown in in FIG. 7 , the APP 1 server determines that the user 1 corresponding to the user identifier 1 likes the actor 1 most, the target voice information in the to-be-pushed notification message corresponding to the user identifier 1 is the voice recorded by the actor 1, and a corresponding resource identifier is a URI 1. The APP 1 server determines that a user 2 corresponding to a user identifier 2 likes the actor 3 most, the target voice information in the to-be-pushed notification message corresponding to the user identifier 2 is a voice recorded by the actor 3, and a corresponding resource identifier is a URI 3.", + "p0127": "In an embodiment of this application, as shown in FIG. 8 , the notification message may usually include one or more of the following: an application icon, an application name, a notification title, notification content, message arrival time, or the like. The notification message forwarded by the APP 1 server to the mobile phone through the push server may be carried by using a specific data structure. For example, the APP 1 server may use a structure in the following JS object notation format for carrying. \"buttons\": }}\"token\" : }}", + "p0128": "The following describes meanings of fields in the structure in sequence.", + "p0129": "An information field is a message root node, and indicates that a message is described.", + "p0130": "An android field indicates a type of an operating system. The message is available in the Android operating system.", + "p0131": "A notification field is a notification message root node, and indicates a notification message displayed in the notification panel on a device side.", + "p0132": "A title field indicates a notification title of the notification message, and is information that can be viewed by the user in a prominent location in the notification panel.", + "p0133": "A body field indicates notification content of the notification message, and is usually followed by the notification title. However, if the title is extremely long, the notification content cannot be completely displayed. In addition, not all notification messages may carry notification content.", + "p0134": "An icon field indicates an application image of a notification, and is used to determine an application icon to be displayed in the notification panel.", + "p0135": "A type field indicates a type of the notification message, for example, a pure text type , an audio type , a video type , or another rich media type.", + "p0136": "A tap action field indicates an opening mode type of a notification after tapping.", + "p0137": "A type field indicates an opening mode type. For example, the URI is opened in a default manner of the operating system, the URI is opened, after tapping, by using an application APP to which the notification belongs, an application APP list is opened after tapping for user selection , or the like.", + "p0138": "A rich media field indicates a rich media storage path or a specified URI.", + "p0139": "A button field indicates an attachment button in the notification panel, and may be used to perform operations, such as \"Learn more\", \"Ignore\", \"Save\", or \"Share\", for the notification in addition to \"Tap\". For another example, the button may be displayed in the notification panel in different forms based on different type 1 types. For example, a play control , a pause control, or the like may be added to an audio.", + "p0140": "A token field is an authorization ticket used for sending a notification message. The authorization ticket is obtained from a ticket server and checked by the push server , to avoid indiscriminate sending of notification messages.", + "p0141": "As shown in FIG. 6 , after receiving the notification message and the device identifier 1 that are sent by the APP 1 server, the push server sends the notification message to the mobile phone 1 corresponding to the device identifier 1. In some embodiments, after receiving the notification message and the device identifier 1 that are sent by the APP 1 server, the push server may determine the APP 1 that applies for the device identifier 1 , and send, in the notification message, an application name of the APP 1 to the mobile phone 1. In some other embodiments, the notification message sent by the APP 1 server to the push server carries an application name of the APP 1 , and the push server sends the notification message and the carried application name of the APP 1 to the mobile phone 1.", + "p0142": "Still as shown in FIG. 6 , after receiving the notification message sent by the push server, the mobile phone 1 displays the notification message on the screen, to display the notification message to the user. Specifically, after receiving the notification message sent by the push server, the mobile phone 1 obtains corresponding voice information based on the resource identifier in the notification message, parses the notification message based on an agreed notification message format, and displays a parsed notification message on the screen. For example, the mobile phone 1 may display the parsed notification message in the notification panel or in another location like at the top of the screen of the mobile phone 1. This is not limited. The notification panel may be invoked by the user by pulling down from the top of the screen, or may be located on a leftmost screen of the mobile phone 1, or may be displayed on a lock screen, or the like. Descriptions are provided below by using an example in which the mobile phone 1 displays the notification message in the notification panel.", + "p0143": "In some embodiments, for the notification message displayed by the mobile phone 1 to the user in the notification panel, refer to FIG. 9 . As shown in FIG. 9 , the notification message includes an application icon 901, an application name 902, and a notification title 903, and may further include prompt information 904 of message arrival time, or other information. In addition, the mobile phone 1 may further add a notification icon or some other information to the notification message.", + "p0144": "The application icon 901 may correspond to the icon field in the foregoing structure, and the notification title 903 may correspond to the title field in the foregoing structure. When the notification message includes voice information, a value of type1 in the type field in the foregoing structure is 2. That is, it indicates that notification content of the notification message is in an audio form.", + "p0145": "In some embodiments, when the notification message includes voice information, the mobile phone 1 may display, in the notification message, a person corresponding to the voice information. For example, as shown in FIG. 9 , on the mobile phone 1, the notification title of the notification message may include person information 90 corresponding to the voice information.", + "p0146": "In some embodiments, when the notification message includes voice information, the mobile phone 1 may prompt, in a specific manner, the user that the current notification message includes the voice information. For example, as shown in FIG. 9 , the notification message displayed on the mobile phone 1 includes a voice identifier 91. For another example, as shown in FIG. 9 , the notification message displayed on the mobile phone 1 includes a voice identifier 92, where the voice identifier is prompt information \"Voice\". In this way, when the notification message includes the voice information, a form and content of the notification message are novel and vivid, and therefore are attractive to the user. In this way, the user is more likely to be interested in playing the voice information in the notification message.", + "p0147": "In some embodiments, the notification message displayed by the mobile phone 1 to the user in the notification panel further includes play duration of the voice information. For example, for the play duration, refer to prompt information 93 shown in FIG. 9 . The prompt information 93 may also be referred to as a voice identifier 93, and may also be for prompting the user that the current notification message includes the voice information.", + "p0148": "In an embodiment of this application, in response to an operation 1 performed by the user on the notification message, the mobile phone 1 plays the voice information included in the notification message. In this way, when the notification message includes the voice information, the notification message is novel and vivid, and therefore is attractive to the user. After playing the voice information in the notification message, the user is likely to get interested and starts the APP, so that usage of the APP is high.", + "p0149": "In addition, the voice information in the notification message is accurately pushed based on a preference of the user, and therefore is attractive to the user. After playing the voice information in the notification message, the user easily gets interested and starts the APP to learn more content, so that usage of the APP is high.", + "p0150": "In some embodiments, in response to the operation 1 performed by the user on the notification message, the mobile phone 1 may directly play the voice information included in the notification message without starting a specific APP to play the voice information. The mobile phone 1 may quickly and conveniently play the voice information in the notification message in a background by using a voice playing basic capability of the operating system. In this case, in the click_action field in the foregoing structure, a value of type2 may be 1. That is, it indicates that the voice information is played in the default manner of the operating system. For example, the mobile phone 1 may play the voice information by invoking the following API interfaces at the bottom layer of the operating system.MediaPlayer mediaPlayer = new MediaPlayer ;//Set a specified streaming media addressmediaPlayer.setDataSource ;//Set an audio stream typemediaPlayer.setAudioStreamType", + "p0151": "In this way, compared with a method that is in the conventional technology and in which the mobile phone displays specific content of the notification message after starting the APP 1 in response to an operation of tapping the notification message by the user, in the method provided in embodiments of this application, the user can directly, quickly, and conveniently play the voice information in the notification message by using the basic capability of the operating system without waiting for a long time to start the APP 1, to obtain specific content of the notification message, and resources required for starting the APP 1 can be reduced.", + "p0152": "In addition, in the conventional technology, in response to the operation of tapping the notification message by the user, after starting the APP 1, the mobile phone finds that the user is not interested in the specific content of the notification message, but spend much time in waiting for starting the APP 1. In this embodiment of this application, the mobile phone may directly play the voice information in the notification message in response to the operation 1 performed by the user on the notification message. If the user is indeed interested in the specific content of the voice information, the user may start the APP 1 for further understanding. If the user is not interested in the voice information, the user does not need to wait for starting the APP 1.", + "p0153": "For example, the resource identifier in the notification message may be a resource link . In an implementation of this application, after receiving the notification message, the mobile phone 1 performs pre-downloading based on the resource link, to obtain corresponding voice information. After a play indication of the user is detected, the voice information obtained through downloading is played. To be specific, after receiving the notification message, the mobile phone 1 parses the notification message based on the agreed format, and downloads specific data content of the voice information from a network based on the resource link in the rich_resource field in the notification message. Then, in response to the operation 1 performed by the user on the notification message, the mobile phone 1 plays the voice information based on the obtained data content. In another implementation, after receiving the notification message, the mobile phone 1 does not immediately pre-download voice information from a network based on the resource link, but downloads and plays the voice information after detecting a play indication of the user. To be specific, in response to the operation 1 performed by the user on the notification message, the mobile phone 1 obtains data content of the voice information corresponding to the resource link in the rich_resource field in the notification message, and plays the voice information included in the notification message.", + "p0154": "For example, the operation 1 performed by the user on the notification message may be an operation performed by the user on the notification title shown in FIG. 10 , the operation performed on the voice identifier in the notification message shown in each of FIG. 9 to FIG. 9 , an operation performed on a blank area in the notification message, or the like. This is not limited. For example, the operation 1 performed by the user on the notification title may be an operation of tapping the notification title by the user. The operation 1 performed by the user on the notification title may alternatively be another touch operation like double-tapping or touching and holding, a voice operation, a gesture operation, or the like. A specific manner of the operation 1 is not limited in embodiments of this application.", + "p0155": "When the mobile phone 1 plays the voice information, the mobile phone 1 may prompt, by displaying prompt information, by using an indicator, or in another manner, the user that the voice information is being played currently. For example, as shown in FIG. 10 , the mobile phone 1 may prompt, by using a prompt identifier 1000, the user that the voice information is being played currently.", + "p0156": "When the mobile phone 1 plays the voice information, if the mobile phone 1 detects the operation 1 again, the mobile phone 1 may pause playing the voice information. Subsequently, after detecting the operation 1 again, the mobile phone 1 continues to play the voice information based on previous play progress. Alternatively, when the mobile phone 1 plays the voice information, if the mobile phone 1 detects the operation 1 again, the mobile phone 1 may stop playing the voice information. Subsequently, after detecting the operation 1 again, the mobile phone 1 replays the voice information.", + "p0157": "In some embodiments, the mobile phone 1 in a screen-locked state plays the voice information in the notification message in response to the operation 1 performed by the user on the notification message. In some other embodiments, the mobile phone 1 can play the voice information in the notification message in response to the operation 1 performed on the user on the notification message only when the mobile phone 1 is in a screen-unlocked state. After the screen is locked, the mobile phone 1 may stop playing the voice information or may continue to play the voice information.", + "p0158": "In some embodiments, the notification message displayed in the notification panel further includes a play control. After detecting an operation performed by the user on the play control, the mobile phone 1 plays the voice information in the notification message. For example, as shown in FIG. 11 , after detecting an operation of tapping a play control 1100 by the user, the mobile phone 1 plays the voice information in the notification message. For another example, the play control may be the foregoing voice identifier. A specific disposing location and a specific form of the play control are not limited in embodiments of this application.", + "p0159": "In some possible implementations, a display status of the play control 1100 in a process of playing the voice information is different from a display status that is of the play control 1100 and that corresponds to a case in which the voice information is not played. For example, the play control 1100 is in a first state when the voice information is not played, and is in a second state in the process of playing the voice information. For example, in the process of playing the voice information, the display status of the play control 1100 may dynamically change as shown in FIG. 11 to FIG. 11 , to prompt the user that the voice information is being played currently.", + "p0160": "In some other possible implementations, a display status of the play control 1100 in a process of playing the voice information is consistent with a display status that is of the play control 1100 and that corresponds to a case in which the voice information is not played.", + "p0161": "In some other possible implementations, after the mobile phone 1 starts to play the voice information, in the notification message displayed in the notification panel, the play control may be switched to a pause control. For example, after detecting an operation of tapping a play control 1200 shown in FIG. 12 by the user, the mobile phone 1 starts to play the voice information, and the pause control 1200 is switched to a pause control 1201 shown in FIG. 12 . If the mobile phone 1 detects an operation of tapping the pause control by the user, the mobile phone 1 may pause playing the voice information, and the pause control is switched to a resume control. Subsequently, after detecting an operation of tapping the resume control by the user, the mobile phone 1 continues to play the voice information based on previous play progress.", + "p0162": "Alternatively, after the mobile phone 1 starts to play the voice information, the play control in the notification message may be switched to a stop control. If the mobile phone 1 detects an operation of tapping the stop control by the user, the mobile phone 1 may stop playing the voice information, and the stop control is switched back to the play control. Subsequently, after detecting the operation of tapping the play control by the user, the mobile phone 1 restarts to play the voice information.", + "p0163": "In some embodiments, in the process of playing the voice information, the mobile phone 1 may further prompt the user with play progress. For example, as shown in FIG. 12 , the mobile phone 1 may display one or more pieces of information such as a play progress bar 1202, total duration 1203, played duration 1204, or remaining play duration of the voice information, to prompt the user with play progress of the current voice information. In some other embodiments, in the process of playing the voice information, the mobile phone 1 may further perform a fastforward or rewind operation in response to an indication operation of the user, to adjust the play progress.", + "p0164": "In some embodiments, in the process of playing the voice information, the mobile phone 1 may further display text content of the voice information to the user on the screen, or display text content of the voice information to the user on the screen in response to an indication operation of the user. The text content may scroll along with the process of playing the voice information. After playing of the voice information is completed, the text content is automatically stopped from being displayed. Alternatively, the text content corresponds to a close control. After detecting an operation of tapping the close control by the user, the mobile phone 1 stops displaying the text content. For example, for the text content of the voice information, refer to text information 1205 shown in FIG. 12 .", + "p0165": "When the mobile phone 1 plays the voice information in the notification message, the notification message is usually slidden to a middle area of the screen. In some embodiments, if the mobile phone 1 moves the notification message to an upper part or a lower part of the screen in response to an operation of sliding up/down on the screen by the user, and moves out of the middle area of the screen , the mobile phone 1 may automatically stop playing the voice information without a user indication. After the notification message is moved back to the middle area of the screen , the mobile phone 1 may automatically continue to play the voice information or replay the voice information without a user indication. This can improve user experience. Alternatively, after the notification message is moved back to the middle area of the screen, the voice information is no longer played.", + "p0166": "In some other embodiments, the notification message displayed by the mobile phone 1 to the user in the notification panel may further include a voice switching control . The voice switching control is for switching between and playing voice information of different persons corresponding to a media resource reserved by the user. The voice switching control corresponds to the buttons field in the foregoing structure. In an implementation, the notification message received by the mobile phone 1 includes resource identifiers of a plurality of pieces of voice information. In response to the operation 1 of the user, the mobile phone 1 preferentially plays voice information of a favorite person of the user. Then, the mobile phone 1 plays other voice information in response to an operation performed by the user on the voice switching control. For example, a sequence of switching between and playing the other voice information by the mobile phone 1 may be sorted based on a preference degree of the user, or other voices between which the mobile phone 1 switches and that are played may be randomly selected by the mobile phone 1 from the plurality of pieces of received voice information. For example, as shown in FIG. 13 , after detecting an operation of tapping a switching control 1300 by the user, the mobile phone 1 plays other voice information corresponding to the movie 1. In addition, as shown in FIG. 13 , the notification title of the notification message also changes correspondingly, and then a changed notification title corresponds to voice information after switching.", + "p0167": "In some other embodiments, in response to an operation 2 performed by the user on the notification message, the mobile phone 1 starts the APP 1 and displays a page of the APP 1. The page of the APP 1 displayed by the mobile phone 1 may be a home page of the APP 1, or may be a page corresponding to the movie 1 corresponding to the voice information, or may be another page. This is not limited. The operation 2 is different from the operation 1. For example, when the operation 1 is the operation of tapping the notification title by the user, the operation 2 is an operation of touching and holding the notification title, an operation of double-tapping the notification title, or the like that is performed by the user. For another example, as shown in FIG. 14 , the operation 2 may be an operation of tapping a name 1401 of the APP 1 or an icon 1402 of the APP 1 by the user. For another example, as shown in FIG. 14 , the notification panel includes a \"Start the application\" control 1403, and the operation 2 is an operation of tapping the control 1403 by the user. In response to the operation 2 of the user, as shown in FIG. 14 , the mobile phone 1 jumps to display the page corresponding to the movie 1 in the APP 1.", + "p0168": "In some other embodiments, as shown in FIG. 15 , the notification message displayed by the mobile phone 1 to the user in the notification panel may further include some other controls such as a Learn more control 1501, an Ignore control 1502, a Save control 1503, or a Share control 1504.", + "p0169": "In response to an operation performed by the user on the Learn more control in the notification message, the mobile phone 1 may start the APP 1 to learn more content related to the movie 1.", + "p0170": "In response to an operation performed by the user on the Ignore control in the notification message, the mobile phone 1 may clear the notification message from the screen, to ignore the notification message. In addition, in response to an operation of sliding left/right on the notification message by the user, the mobile phone 1 may clear the notification message from the screen, to ignore the notification message.", + "p0171": "In response to an operation performed by the user on the Save control in the notification message, the mobile phone 1 may save the voice information in the notification message, to subsequently play the voice information again or share the voice information with another object .", + "p0172": "The mobile phone 1 may share the voice information with the another object in response to an operation performed by the user on the Share control in the notification message. For example, the mobile phone 1 shares the voice information with another contact by using an APP , shares the voice information on Moments, or shares the voice information on another public platform. Alternatively, the user may choose to share the voice information with the another device by using a network or in a transmission manner. For example, in response to an operation of tapping the Share control 1504 shown in FIG. 15 by the user, the mobile phone 1 may display a sharing list 1505 shown in FIG. 15 , so that the user chooses, from the sharing list 1505, to share the voice information with the another object.", + "p0173": "In some other embodiments, the mobile phone 1 may display, in a centralized manner, a plurality of notification messages pushed by a same APP server. For example, as shown in FIG. 16 , each notification message may still be separately displayed as one card. Alternatively, the plurality of notification messages may be combined in one card for display. For example, as shown in FIG. 16 and FIG. 16 , two notification messages pushed by the APP 1 may be combined for display, and a notification message after combination includes two notification titles respectively corresponding to the two notification messages before the combination. In an implementation, as shown in FIG. 16 , each notification title corresponds to one play control for playing corresponding voice information. In another implementation, as shown in FIG. 16 , the notification message after the combination includes one play control. After detecting an operation of tapping the play control by the user, the mobile phone 1 sequentially plays each piece of voice information in the notification message after the combination. In addition, a quantity of included messages may be further displayed in the notification message after the combination.", + "p0174": "In some other embodiments, the notification panel includes a global play control. The mobile phone 1 may sequentially play voice information in the plurality of notification messages in the notification panel in response to an operation performed by the user on the global play control.", + "p0175": "In some embodiments, in addition to the voice information, the notification message displayed by the mobile phone 1 to the user in the notification panel further includes notification content. The notification content may be text information corresponding to the voice information, a keyword or summary information corresponding to the voice information, or the like. The notification content may correspond to the body field in the foregoing structure. Alternatively, the notification content of the notification message includes the voice information, and the text information, the keyword, the summary information, or the like corresponding to the voice information.", + "p0176": "In some other embodiments, the APP 1 server may also send, to the mobile phone 1 by using a notification message, voice information of a plurality of associated persons corresponding to the movie 1. The mobile phone 1/A cloud server determines a person that the user likes, and then displays, to the user in the notification panel, a notification message including a voice of the person that the user likes, but no longer displays, to the user, a notification message including a voice of another person. For example, the mobile phone 1/cloud server may determine the person that the user likes based on user behavior information in a process of using the mobile phone 1 by the user. For example, the mobile phone 1 determines that videos related to a person 1 are played the largest quantity of times when the user plays videos by using various video playing APPs, to determine that the user likes the person 1. Then, the mobile phone 1 displays, to the user, a notification message including a voice of the person 1.", + "p0177": "In some other embodiments, after detecting the operation 1 performed by the user on the notification message, the mobile phone 1 starts the APP 1, and plays the voice information in the notification message by using the APP 1. In this case, in the click_action field in the foregoing structure, a value of type2 may be 2. That is, it indicates that the voice information is played by using the APP 1 corresponding to the notification message. For example, after the mobile phone 1 detects an operation of tapping the play control in the notification message shown in FIG. 17 by the user, as shown in FIG. 17 , the mobile phone 1 starts the APP 1 and plays the voice information in the notification message by using the APP 1. After starting the APP 1, the mobile phone 1 may automatically start to play the voice information, or may start to play the voice information after detecting an operation of a play indication of the user. After playing of the voice information is completed, the mobile phone 1 may continue to stay in the APP 1, so that the user can play another audio or watches a movie; or may return to a previous interface by using a return control, a close control, or the like.", + "p0178": "In some other embodiments, after detecting the operation 1 performed by the user on the notification message, the mobile phone 1 displays a play mode list for user selection. This can improve flexibility of playing the voice information. For example, after the mobile phone 1 detects an operation of tapping the play control in the notification message shown in FIG. 18 by the user, as shown in FIG. 18 , the mobile phone 1 displays the play mode list of the voice information. The play mode list may include a play mode 1801 of a default audio player of a system, a play mode 1802 of the current APP 1, and a play mode 1803 of another audio player like an APP 2 supported by the system. This facilitates user selection. The default audio player of the system, the current APP 1, and the another audio player supported by the system are all applications. In other words, the play mode list includes a plurality of applications for playing the voice information. In this case, in the click_action field in the foregoing structure, a value of type2 may be 3. That is, it indicates that the user can select a play mode of the voice information from the list. In response to an operation of selecting the play mode 1801 shown in FIG. 18 by the user, the mobile phone 1 plays the voice information in the notification message by using an operating system basic capability of the mobile phone 1. In response to an operation of selecting the play mode 1802 shown in FIG. 18 by the user, the mobile phone 1 plays the voice information in the notification message by using the APP 1. In response to an operation of selecting the play mode 1803 shown in FIG. 18 by the user, as shown in FIG. 18 , the mobile phone 1 starts the APP 2, displays a page of the APP 2, and plays the voice information in the notification message by using the APP 2 corresponding to the play mode 1803. After playing of the voice information is completed, the mobile phone 1 may continue to stay in the APP 2, so that the user can play another audio or watches a movie; or may return to a previous interface by using a return control, a close control, or the like.", + "p0179": "The mobile phone 1 may identify a file type based on information such as a file suffix or a file name of the resource identifier. The file type is a text, an audio, a video, or another format. For example, the resource identifier is a URI, and a file suffix of the URI is \".mp3\". In this case, the mobile phone 1 identifies, based on the suffix, that the file type is an audio. The mobile phone 1 obtains, based on a configuration file of an operating system, a list of supported APPs, such as Storm Player \u00ae and MX Player \u00ae , corresponding to the file type. The mobile phone 1 displays a play mode list that supports opening the file type. The play modes include Storm Player, MX Player, and an audio player provided by the system. After the user selects a play mode like Storm Player, the mobile phone 1 starts Storm Player and transfers the URI, and then Storm Player accesses the URI to obtain corresponding voice information for playing.", + "p0180": "In some embodiments, when displaying the play mode list for user selection, the mobile phone 1 may further prompt the user whether the voice information is always played in the play mode or the voice information is played in the play mode only once.", + "p0181": "In some other embodiments, when the mobile phone 1 plays the voice information in the notification message in a specific play mode, the user may select or switch a play mode type of the voice information in the notification message. This can improve flexibility of playing the voice information. In this case, in the click_action field in the foregoing structure, a value of type2 may be empty; or a value of type2 may be 1 or 2 or 3. In a possible implementation, an interface of a system setting application may include a play mode type list. The type list includes a type of the default audio player of the system, a type of an APP corresponding to the notification message, a type of the play mode list, and the like. The user can specify, in the system setting application, a play mode type of the voice information in the notification message. After detecting the operation 1 performed by the user on a notification message 1, the mobile phone 1 plays the voice information by using the play mode type specified by the user.", + "p0182": "In some other embodiments, the structure for carrying the notification message includes the click_action field, and the user customizes a play mode of the voice information on the mobile phone 1. In this case, in an implementation, the voice information is played in a play mode specified by the click_action field. In another implementation, the voice information is played in the play mode customized by the user on the mobile phone 1.", + "p0183": "In some other embodiments, the structure for carrying the notification message does not include the click_action field. The mobile phone 1 may play the voice information in the notification message in the play mode customized by the user on the mobile phone 1.", + "p0184": "In another possible implementation, if the mobile phone 1 detects an operation of tapping the notification title/play control by the user, the voice information is played by using the play mode type indicated by the click_action field in the structure. If the mobile phone 1 detects an operation of touching and holding the notification title/play control by the user, the play mode type list is played, so that the user switches to another play mode in the list for playing.", + "p0185": "In another possible implementation, the notification message displayed by the mobile phone 1 to the user in the notification panel includes a play type control/type switching control. The mobile phone 1 may display the play mode type list in response to an operation performed by the user on the play type control/type switching control, so that the user switches to another play mode in the list for playing. In addition, the mobile phone 1 may further prompt the user with the currently used play mode of the voice information in the notification message. For example, as shown in FIG. 19 , the mobile phone 1 prompts, by using a control 1901, the user that the voice information in the notification message is played in the background currently in a system default mode. After detecting an operation of tapping a type switching control 1902 by the user, as shown in FIG. 19 , the mobile phone 1 displays the play mode type list of the voice information.", + "p0186": "Descriptions are provided above by using an example in which the notification message includes rich media information, and the rich media information includes voice information . In some other embodiments, the notification message may further include rich media information of another type, for example, a notification message including video information or a notification message including animated image information . Similarly to the case that the notification message includes the voice information, when the notification message includes the video information, the method provided in the foregoing embodiments of this application may still be used to push the message. A difference is described below.", + "p0187": "In some embodiments, the notification message displayed by the mobile phone 1 to the user in the notification panel may further include prompt information of the video information. For example, the prompt information is a thumbnail 2000 shown in FIG. 20A . The mobile phone 1 may play the video information in the notification message in response to an operation performed by the user on the thumbnail 2000.", + "p0188": "Similar to playing the voice information in the notification message, the mobile phone 1 may play the video information in the notification message in a default video play mode of the system, by using the APP 1 for pushing the notification message, or by using another application that can play a video and that is supported by the system.", + "p0189": "In an embodiment of this application, when the mobile phone 1 plays the video information, the video information may be played in a small window in the notification message on the screen ), or may be played in a large window ), or may be played in a full-screen window ). This is not limited. In response to an operation of the user, the mobile phone 1 may switch between the windows for playing the video information in the notification message. In addition, a play window may further include a pause control, a next control, a play progress bar, a full-screen/small-window switching control, and the like.", + "p0190": "In some embodiments, after playing of the video information in the notification message is completed, the mobile phone 1 automatically closes the play window.", + "p0191": "In some other embodiments, after playing of the video information in the notification message is completed, the mobile phone 1 does not automatically close the play window, and the pause control may be switched to a play control/replay control. If duration in which the playing is stopped is greater than preset duration, the play window is automatically closed. Alternatively, the play window needs to be manually closed by the user.", + "p0192": "For example, in response to an operation performed by the user on the thumbnail 2000 shown in FIG. 20A , the mobile phone 1 may display a playlist shown in FIG. 20B . The play mode list may include a play mode 2001 of a default video player of the system, a play mode 2002 of the current APP 1, and a play mode 2003 of another video player like the APP 2 supported by the system. This facilitates user selection. The default video player of the system, the current APP 1 to which the notification message is pushed, and the another video player supported by the system are all applications. In other words, the play mode list includes a plurality of applications for playing the video information. In response to an operation of selecting the play mode 2001 shown in FIG. 20B by the user, the mobile phone 1 plays the video information in the notification message by using an operating system basic capability of the mobile phone 1. In response to an operation of selecting the play mode 2002 shown in FIG. 20B by the user, the mobile phone 1 plays the video information in the notification message by using the APP 1. In response to the operation of selecting the play mode 2002 shown in FIG. 20B by the user, as shown in FIG. 20B , the mobile phone 1 starts the APP 1 , displays a page of the APP 1, and plays the video information in the notification message by using the APP 1 corresponding to the play mode 2001. After starting the APP 1, the mobile phone 1 may automatically start to play the video information, or may start to play the video information after detecting an operation of a play indication of the user. After playing of the video information is completed, the mobile phone 1 may continue to stay in the APP 1, so that the user can play another video or watches a movie; or may return to a previous interface by using a return control, a close control, or the like.", + "p0193": "A case in which the mobile phone 1 plays the video information in the notification message by using the APP 2 is similar to the case in which the mobile phone 1 plays the video information in the notification message by using the APP 1. Details are not described.", + "p0194": "Descriptions are provided above by using an example in which message push is triggered after the user reserves the movie 1. When the APP 1 pushes the notification message on another occasion, the method provided in the foregoing embodiments may also be used to push the message. For example, in a non-reservation scenario, the APP 1 server may push, to the mobile phone, a notification message corresponding to a to-be-pushed media resource . The to-be-pushed media resource may correspond to a plurality of pieces of rich media information . The APP 1 server may determine, based on a user feature, rich media information corresponding to a favorite associated person of the user, and push the determined rich media information to the user in the notification message.", + "p0195": "Descriptions are provided above by using an example in which a message is pushed after the user logs in to the APP 1. When the user does not log in to the APP 1, the method provided in the foregoing embodiments may also be used to push the message. For example, when the user does not log in to the APP 1 shown in FIG. 5A , the APP 1 server may accurately determine, based on a device identifier of the electronic device, target voice information to be pushed to the electronic device corresponding to the device identifier, to push a personalized message. For example, the APP 1 server may accurately determine, based on an area and a location of the electronic device that correspond to the device identifier, or a user feature of a user using the electronic device, the target voice information to be pushed to the electronic device corresponding to the device identifier.", + "p0196": "Descriptions are provided above by using an example in which the APP 1 server accurately pushes, to the user, a notification message related to the movie 1. The APP server may also accurately push, to the user based on the user behavior information, the user profile, or the like, a notification message corresponding to a movie type that the user is interested in. The movie type is, for example, a comedy movie, a suspense movie, or a historical movie.", + "p0197": "In addition, descriptions are provided above by using an example in which the APP 1 corresponding to the notification message is a video playing APP. The APP 1 may alternatively be an application of another type, and the method provided in embodiments of this application may also be used to accurately push the notification message. In this way, the user may receive a notification message that the user is interested in, and therefore is more willing to open the notification message and start a corresponding APP. For example, a news APP server may accurately push a news notification message to the user based on user behavior information, a user profile, or the like. For example, the user views daily news in a specific news APP. The APP server identifies, based on a watching record of the user, a news type that the user is interested in, for example, sports news + entertainment news. The APP server periodically pushes the latest and hot news of a current day every day. In this case, the APP server selects sports and entertainment notification messages based on an interest point of the user. For another example, the APP server accurately pushes a military notification message, a sports notification message, a health care notification message, or the like based on a preference of the user. The APP server sends notification content of a selected notification message to the user in a form of a voice.", + "p0198": "In an embodiment of this application, the notification message displayed by the mobile phone in the notification panel may be a notification message including the rich media information, or may be a text notification message that does not include the rich media information. In addition, different notification messages may further include different types of rich media information. For example, as shown in FIG. 21 , the mobile phone 1 displays notification messages of a plurality of applications to the user in the notification panel on the screen, such as a voice notification message 2101 of a Huawei Video application, a voice notification message 2102 of a News application, a video notification message 2103 of a Storm Player \u00ae application, and a text notification message 2104 of a Browser application. The voice notification message 2101 is voice information that is of the actor 1 and that is accurately recommended to the user, and the voice notification message 2102 is a sports notification message accurately recommended to the user.", + "p0199": "In some other embodiments of this application, the user may set, on a system setting interface or on a setting interface of an application, a type of a notification message pushed by the application, for example, a voice notification message, a video notification message, an animated image notification message, a text notification message, or another type of notification message.", + "p0200": "In some other embodiments of this application, in response to an operation of the user, the mobile phone 1 may switch, to a conventional text notification message, the notification message that includes the rich media information and that is displayed to the user on the screen; or switch between a text notification message, a voice notification message, a video notification message, an animated image notification message, and another type of notification message.", + "p0201": "Descriptions are provided below by using an example in which the electronic device is the mobile phone. When the electronic device is a tablet computer or another device, the method described in the foregoing embodiments may still be used to push a message. Details are not described again.", + "p0202": "With reference to the foregoing embodiments and the corresponding accompanying drawings, another embodiment of this application provides a notification message processing method. The method may be implemented on the electronic device having the structure shown in FIG. 1 . As shown in FIG. 22 , the method includes the following steps.", + "p0203": "2201: An electronic device obtains a target notification message of a target application, where the target notification message includes first rich media information, and the first rich media information includes one or more of voice information, video information, or an animated image.", + "p0204": "A server of the target application may forward the target notification message to the electronic device through the push server, and the electronic device may obtain the target notification message of the target application. The target application may be any application like a system application or a third-party application, for example, may be a video playing application, an audio playing application, a music playing application, a news application, a game application, an e-book application, or a financial application. The target notification message includes the first rich media information, and the first rich media information includes one or more of the voice information, the video information, or the animated image.", + "p0205": "For example, refer to the foregoing embodiments. The target application is Huawei Video, and the first rich media information includes the voice information.", + "p0206": "2202: The electronic device displays a first interface, where the first interface includes notification messages of at least two applications, the at least two applications include the target application, and the notification messages of the at least two applications include the target notification message.", + "p0207": "The electronic device may display, to a user on a screen, notification messages that are of a plurality of applications and that include the target notification message of the target application.", + "p0208": "For example, the target notification message may be the notification message pushed by Huawei Video shown in FIG. 9 to FIG. 21 , rich media information included in the target notification message is voice information, and the voice information is the voice information of the actor 1.", + "p0209": "2203: The electronic device plays the first rich media information in the target notification message in response to a trigger operation performed by the user on the target notification message.", + "p0210": "The electronic device may play the first rich media information in the target notification message in response to the trigger operation performed by the user on the target notification message. For example, the trigger operation may be an operation of tapping a notification title in the target notification message by the user, an operation of tapping a play control, an operation of tapping a rich media information identifier , or the like that is performed by the user.", + "p0211": "In this solution, the electronic device can push, to the user, rich media information such as a voice in a notification message pushed by an APP, so that a form and content of the notification message are novel and vivid, and therefore are attractive to the user. After playing the rich media information such as the voice in the notification message, the user is more likely to get interested and starts the APP. This can improve usage of the APP.", + "p0212": "It may be understood that, to implement the foregoing functions, the electronic device includes a corresponding hardware and/or software module for performing each function. With reference to algorithm steps in the examples described in embodiments disclosed in this specification, this application can be implemented by hardware or a combination of hardware and computer software. Whether a function is performed by hardware or hardware driven by computer software depends on particular applications and design constraints of the technical solutions. A person skilled in the art may use different methods to implement the described functions for each particular application with reference to embodiments, but it should not be considered that the implementation goes beyond the scope of this application.", + "p0213": "In this embodiment, the electronic device may be divided into functional modules according to the foregoing method examples. For example, each functional module may be obtained through division based on each corresponding function, or two or more functions may be integrated into one module. The integrated module may be implemented in a form of hardware. It should be noted that, in embodiments, division into the modules is an example, is merely logical function division, and may be other division during actual implementation.", + "p0214": "When each function module is obtained through division based on each corresponding function, in an example, the electronic device may include an obtaining unit, a display unit, a play unit, a processing unit, and the like. It should be noted that all related content of the steps in the foregoing method embodiments may be cited in function descriptions of corresponding functional modules. Details are not described herein again.", + "p0215": "An embodiment of this application further provides an electronic device. As shown in FIG. 23 , the electronic device includes a screen 2301, one or more processors 2302, a memory 2303, and one or more computer programs 2304. The foregoing components may be connected through one or more communication buses 2305. The one or more computer programs 2304 are stored in the memory 2303, and are configured to be executed by the one or more processors 2302. The one or more computer programs 2304 include instructions, and the instructions may be used to perform steps in the foregoing embodiments. All related content of the steps in the foregoing method embodiments may be cited in function descriptions of corresponding physical components. Details are not described herein again.", + "p0216": "For example, the processor 2302 may be specifically the processor 110 shown in FIG. 3 , the memory 2303 may be specifically the internal memory 121 shown in FIG. 3 , and the screen 2301 may be specifically the display 194 shown in FIG. 3 .", + "p0217": "An embodiment of this application further provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors. The one or more memories are configured to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the electronic device is enabled to perform the foregoing related method steps to implement the notification message processing method in the foregoing embodiments.", + "p0218": "An embodiment of this application further provides a server, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors. The one or more memories are configured to store computer program code, and the computer program code includes computer instructions. When the one or more processors execute the computer instructions, the server is enabled to perform the foregoing related method steps to implement the notification message processing method in the foregoing embodiments.", + "p0219": "An embodiment of this application further provides a computer-readable storage medium. The computer-readable storage medium stores computer instructions. When the computer instructions are run on an electronic device or a server, the electronic device or the server is enabled to perform the foregoing related method steps to implement the notification message processing method in the foregoing embodiments.", + "p0220": "An embodiment of this application further provides a computer program product. When the computer program product runs on a computer, the computer is enabled to perform the foregoing related steps to implement the notification message processing method performed by the electronic device in the foregoing embodiments.", + "p0221": "In addition, an embodiment of this application further provides an apparatus. The apparatus may be specifically a chip, a component, or a module. The apparatus may include a processor and a memory that are connected. The memory is configured to store computer-executable instructions. When the apparatus runs, the processor may execute the computer-executable instructions stored in the memory, so that the apparatus performs the notification message processing method performed by the electronic device or the server in the foregoing method embodiments.", + "p0222": "The electronic device, the server, the computer-readable storage medium, the computer program product, or the apparatus provided in embodiments is configured to perform the corresponding method provided above. Therefore, for beneficial effects that can be achieved, refer to the beneficial effects of the corresponding methods provided above. Details are not described herein again.", + "p0223": "The foregoing descriptions about implementations allow a person skilled in the art to understand that, for a purpose of convenient and brief description, division of the foregoing functional modules is taken as an example for illustration. During actual application, the foregoing functions may be allocated to different functional modules for implementation based on a requirement. In other words, an inner structure of an apparatus is divided into different functional modules to implement all or some of the functions described above.", + "p0224": "In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. For example, the described apparatus embodiment is merely an example. For example, the module or unit division is merely logical function division and may be other division during actual implementation. For example, a plurality of units or components may be combined or integrated into another apparatus, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented by using some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic, mechanical, or other forms.", + "p0225": "The units described as separate parts may or may not be physically separate, and parts displayed as units may be one or more physical units, may be located in one place, or may be distributed on different places. Some or all of the units may be selected based on an actual requirement to achieve the objectives of the solutions in embodiments.", + "p0226": "In addition, functional units in embodiments of this application may be integrated into one processing unit, each of the units may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in a form of hardware, or may be implemented in a form of a software functional unit.", + "p0227": "When the integrated unit is implemented in the form of the software functional unit and sold or used as an independent product, the integrated unit may be stored in a readable storage medium. Based on such an understanding, the technical solutions of this application essentially, or the part contributing to the conventional technology, or all or some of the technical solutions may be implemented in the form of a software product. The software product is stored in a storage medium and includes several instructions for instructing a device or a processor to perform all or some of the steps of the methods described in embodiments of this application. The foregoing storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory , a random access memory , a magnetic disk, or an optical disc.", + "p0228": "The foregoing content is merely specific implementations of this application, but is not intended to limit the protection scope of this application. Any variation or replacement readily figured out by a person skilled in the art within the technical scope disclosed in this application shall fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.", + "features": { + "F1": "The method involves obtaining a target notification message of a target application by an electronic device, where the message includes first rich media information comprising voice information, video information, or an animated image.", + "F2": "The electronic device displays a first interface that includes notification messages from at least two applications, including the target notification message.", + "F3": "The method involves playing the first rich media information in the target notification message by the electronic device in response to a user-triggered operation on the target notification message.", + "F4": "On the first interface, the target notification message includes a prompt identifier that indicates the presence of rich media information.", + "F5": "The playing of the first rich media information involves tapping the prompt identifier in the target notification message by the user.", + "F6": "The target notification message includes a notification title, and playing is triggered by a user tapping the notification title.", + "F7": "The target notification message includes a play control that can be tapped by the user to play the first rich media information.", + "F8": "The first rich media information is determined by a server based on a user feature that includes user behavior information or a user profile.", + "F9": "The first rich media information is one of multiple pieces of rich media information corresponding to a target media resource, chosen based on the user feature.", + "F10": "The target media resource is a media resource reserved by the user, and the first rich media information corresponds to a favorite associated person of the user.", + "F11": "On the first interface, the target notification message includes first prompt information indicating the play duration of the rich media information.", + "F12": "The method includes displaying second prompt information to indicate the play progress of the rich media information.", + "F13": "The target notification message includes a switching control for playing second rich media information in response to a user operation.", + "F14": "On the first interface, the prompt identifier is in a first state, and in a second interface, it is in a second state, which is different from the first state.", + "F15": "Playing the first rich media information may involve displaying a third interface, which is an application interface of the target application.", + "F16": "The target notification message includes prompt information of the target application like an icon and/or name, used to display the third interface.", + "F17": "The third interface can be displayed by touching and holding the notification title in the target notification message.", + "F18": "The method includes displaying a play mode list upon a user-triggered operation, allowing selection of an application to play the rich media information.", + "F19": "The electronic device stores a correspondence preset by the user between an application and a rich media information type.", + "F20": "The target notification message includes a resource identifier for the first rich media information, which is used to obtain and play the media.", + "F21": "The target notification message may include identification information, icon information, notification title, message type, or device identifier.", + "F22": "In the server method, a target notification message with first rich media information is generated and pushed to an electronic device.", + "F23": "The server determines the first rich media information from multiple pieces of information based on a user feature for the target media resource.", + "F24": "An electronic device includes a screen, processor, memory, and computer programs that enable the notification message processing method.", + "F25": "The computer-readable storage medium contains instructions that enable a computer to perform the notification message processing method.", + "F26": "The computer program product enables a computer to perform the notification message processing method when executed." + } + } + }, + { + "FAN": "79314580", + "CPC": [ + "G06F-016/00", + "G06F-016/245", + "G06F-016/338", + "G06F-017/30/696", + "G06Q-010/04", + "G06Q-010/06/3116", + "G06Q-010/06/316", + "G06Q-010/06/33", + "G06Q-010/10/3", + "G09B-019/00/3", + "G09B-019/00/92" + ], + "Content": { + "title": "Meal preparation orchestrator", + "pa01": "Control unit and method for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes stored in a database on a common recipe format, by independent recipe authors . The method comprises offering dishes to a user, which correspond to a respective recipe in the database ; receiving a selection of dishes from the user; determining a required work intensity of the user, for each recipe step; determining a maximum time limit of each step of the respective recipes ; composing a time interval structure of the steps of the respective recipes , based on the required work intensity and the maximum time limit of each step; and outputting the composed time interval structure to the user.", + "c-en-0001": "A method of cooking a meal, comprising: providing a cooking device in wireless communication with a control unit, the cooking device being in wireless communication with the control unit, the control unit being in communication with a database, a comparison unit and a communication device; storing a first cooking recipe in a first format in the database, the first format containing a resource segment, a cooking step or task segment, each cooking step segment containing a work intensity, a minimum time, a base time and a maximum time, the base time being same or longer than the minimum time and same or shorter than the maximum time, the first cooking recipe requiring a first resource and at least a first cooking step and a second cooking step; storing a second cooking recipe in the first format in the database, the second cooking recipe requiring a second resource and at least a first cooking step and a second cooking step, the first and second cooking recipes being part of a meal; the communication device sending a request signal to the control unit, the request signal containing a request to plan the meal requiring at least the first cooking recipe and the second cooking recipe and a completion time of the meal; the control unit sending a retrieve signal to the database to retrieve the first cooking recipe and the second cooking recipe; the control unit sending a retrieve signal to the communication device to retrieve a completion time of the meal; the control unit adding up the base times of the first cooking step and the second cooking step of the first cooking recipe to determine a first total time required to complete the first cooking recipe; the control unit adding up the base times of the first cooking step and the second cooking step of the second cooking recipe to determine a second total time required to complete the second cooking recipe; the control unit determining a starting time based on the first total time, the second total time and the completion time; the control unit adding up the work intensity of timewise overlapping cooking steps from the first recipe and the second recipe to a combined work intensity for preparing the meal; when the first combined work intensity for the first cooking step of the first recipe and the first cooking step of the second recipe exceeds 100%, the control unit sending a control signal to the comparison unit to compare the starting time of the first cooking step of the first recipe with the starting time of the first cooking step of the second recipe and to select the recipe with the first cooking step having an earliest starting time, the comparison unit sending a control signal to the control unit to advance the completion time of the first cooking step for the selected recipe having the earliest starting time for the first cooking step and to delay the starting time of the first cooking step for the recipe having the latest starting time of the first cooking step; the control unit iteratively advancing the completion time of the first cooking step for the selected recipe with the earliest starting time by reducing the base time for this cooking step and increasing the base time for the second cooking step of the selected recipe and delaying the starting time of the first cooking step for the recipe with the latest starting time by reducing the base times of the first cooking step and the second cooking step for this recipe until the combined work intensity no longer exceeds 100%; when the first combined work intensity for the first cooking step of the first recipe and the first cooking step of the second recipe exceeds 100% and all minimum time limits and all maximum time limits are reached, the control unit sending a message to the communication unit to choose a new combination of recipes for the meal; when the combined work intensity remains at a maximum 100% from the starting time to the completion time, the control unit prioritizing the cooking steps having a lowest work intensity when two or more cooking steps are scheduled to start at the same time; the control unit generating and transmitting control signals to the communication device of the user or to a cooking device for outputting the composed time interval structure on the communication device and/or the cooking device; the control unit receiving at least one input value from a sensor or from the communication device; the control unit generating and transmitting control signals to the communication device in response to the input value from the sensor for outputting the adapted time interval structure on the communication device and/or the cooking device.", + "c-en-0002": "The method according to claim 1 wherein the method further comprises the step of the control unit sending a comparison signal to the comparison unit to compare the work intensity of the second cooking step of the first cooking recipe with the first cooking step of the second cooking recipe, the control unit sending an instruction signal to the communication device to start the second cooking step of the first cooking recipe before the first cooking step of the second cooking recipe when the second cooking step of the first cooking recipe has a work intensity lower than the work intensity of the first cooking step of the second cooking recipe.", + "c-en-0003": "The method according to claim 1 wherein the method further comprises the step of providing a sensor on the cooking device, the sensor sensing a temperature on the cooking device, the sensor sending a temperature signal to the control unit when the temperature approaches, reach or exceeds a threshold value.", + "c-en-0004": "The method according to claim 3 wherein the method further comprises the step of the control unit receiving the temperature signal at a first time and comparing the first time to a reference time on a countdown timeline, the control unit sending a warning signal to the communication device when the first time is later than the reference time.", + "c-en-0005": "The method according to claim 4 wherein the method further comprises the step of the control unit receiving the temperature signal at a first time and comparing the first time to a reference time for completing a cooking step and sending a warning signal to the communication device when the first time exceeds a maximum time set for the cooking step.", + "c-en-0006": "The method according to claim 1 wherein the method further comprises the step of starting a cooking step of the first cooking recipe before a cooking step of the second cooking recipe when the cooking step of the first cooking recipe has a work intensity that is lower than the work intensity of the cooking step of the second cooking recipe.", + "c-en-0007": "The method according to claim 1 wherein the method further comprises the step of the sensor sending information signals directly to the communication device.", + "c-en-0008": "The method according to claim 1 wherein the method further comprises the step of the control unit comparing resources required for the first cooking recipe with the resources required for the second cooking recipe, further identifying and resolving a conflict in resource utilization.", + "c-en-0009": "The method according to claim 1 wherein the method further comprises the step of the control unit adjusting an permitted preparation time of cooking steps so that the first and second cooking recipes are completed at the same time.", + "c-en-0010": "The method according to claim 1 wherein the method further comprises the step of the control unit displaying each subsequent cooking step of the first and second cooking recipe along the countdown timeline so that a second cooking step of the first cooking recipe and the second cooking step of the second cooking recipe are simultaneously completed.", + "p0001": "PRIOR APPLICATION", + "p0002": "This is a continuation-in-part patent application that claims priority from U.S. patent application Ser. No. 15/713,902, filed 25 Sep. 2017 that claims priority from Swedish patent application no. 1600281-8, filed 7 Oct. 2016.", + "p0003": "TECHNICAL FIELD", + "p0004": "Implementations described herein generally relate to a control unit and a method. More particularly, a method and a control unit are described, for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes stored in a database on a common recipe format, by independent recipe authors.", + "p0005": "BACKGROUND", + "p0006": "Recipes for various dishes may be available on an electronic format, e.g. accessible via internet or an app and typically provide information concerning which ingredients to utilise, which quantities to apply, and instructions concerning the dish preparation.", + "p0007": "However, these recipes on the electronic format have not evolved much beyond the classical concept of recipes provided in paper format, i.e. the user has to read a lot of text in order to utilise the recipe and understand which utensils and ingredients to use, and the preparation order of the various moments and the possible advantages of digital recipes are yet not exploited fully.", + "p0008": "Further, when preparing a meal, typically a plurality of dishes has to be cooked and thus also a plurality of recipes has to be used simultaneously. However, there is in general no advice provided to the user, how to combine the different work moments of the different recipes in order to get the different dishes ready at the same time and/or in the shortest possible time, except possibly in the very specific case when one chef/author has written a specific recipe including several dishes to be composed into one meal without possibility for the user to alter any of the dishes due to personal taste, allergy or similar.", + "p0009": "Thus, there may exist recipes for complete meals wherein the preparation order is set and organised by the chef/author, but in case the user would like to compose a meal out of dishes/recipes written by distinct authors, the user has to figure out him/herself in which order to make the different method steps of the respective recipes in order to be ready at dinner time and/or as quickly as possible.", + "p0010": "Normally, there is no possibility for the chef/author of a recipe to specify how long time the dish may be made in advance before deteriorating, other than plain text within the recipe. When preparing a plurality of dishes for a meal, it may often be necessary, in particular when two or more dishes require the same resource and/or the user's complete attention, to prepare the dishes sequentially and the inexperienced user may not know which dish to start prepare and then leave to rest while preparing the other dish/-es; and for how long time it may rest before deteriorating.", + "p0011": "The user, although possibly inexperienced in cooking and/or having problems reading recipes may yet have developed a progressive or unconventional taste; and/or have an opened and experimental mindset and for example be willing to exchange the customary boiled potatoes of a Scandinavian meal such as meatballs with lingonberry jam, with an exotic vegetable such as e.g. cooking plantain, yams and/or cassava. It is then far from obvious for the typical user how to prepare the exotic vegetable, how to organise various cocking moments of the vegetable in combination with the preparation of the meatballs and/or how long time in advance the dish may be cooked/kept warm before losing its freshness, for example. It is also a very small probability that the user finds a readymade meal preparation made by a chef/author in this kind of daring food combinations.", + "p0012": "The user may further be interested in preparing a certain set of dishes when inviting his/her friends; however due to various allergies, diets, phobias, etc. among the guests, he/she may have to prepare alternative replacement dishes comprising edibles with which he/she is not very accustomed. Further, the thereby increased number of dishes creates increased problems for the user concerning how to organise the food preparation, in which order to perform the various steps of the respective recipes; and to determine how to handle conflicts in resource utilisation when preparing different dishes; several dishes may require preparation in the oven at different temperatures and length, for example.", + "p0013": "In case the user makes a search for recipes on the internet, or a particular recipe provider via internet, in order to discover and combine different recipes into a meal, the user most likely will be obliged to read a substantial amount of texts and him/herself combine the various recipe steps into a working list; which is problematic not only for the dyslectic user ; the reader with reading disability , see https://sv.wikipedia.org/wiki/Pisa_ ; or people with age-related alteration of the eye lens, who are required to firstly retrieve their reading glasses before preparing any action; but most likely for any user as cooking typically involve wet/sticky fingers, which combines badly with a computer/touchscreen communication interface and the requirement of text scrolling and recipe swapping.", + "p0014": "It would be desired to provide the inexperienced kitchen user with a tool, enabling him/her to prepare a time planned meal with a flexibility to use recipes of choice beyond what is currently found in plain text recipes or today available tools on the market.", + "p0015": "SUMMARY", + "p0016": "It is therefore an object to obviate at least some of the above mentioned disadvantages and to provide assistance to a user when preparing a meal.", + "p0017": "This and other objects are achieved by the features of the appended independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures.", + "p0018": "According to a first aspect, a control unit is provided. The control unit aims at assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes stored in a database on a common recipe format, by independent recipe authors. The control unit is configured to offer a plurality of dishes to the user via a user device, where each dish corresponds to a respective recipe in the database. Further, the control unit is configured to receive a selection of dishes forming the meal, made by the user, via a communication interface. The control unit is in addition configured to determine a required work intensity for/from the user, for each recipe step in a respective recipe associated with the received dish selection, based on information provided by the recipe author. The control unit is also configured to determine a maximum time limit of each recipe step of the respective recipe, based on information provided by the recipe author. Furthermore, the control unit is additionally configured to compose a time interval structure of the recipe steps of the respective recipe, based on the determined required work intensity for/from the user for each recipe step and the determined maximum time limit of each recipe step. The control unit is configured to generate control signals to output the composed time interval structure to the user device of the user.", + "p0019": "According to a second aspect, a method is provided. The method aims at assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes stored in a database on a common recipe format, by independent recipe authors. The method comprises offering a plurality of dishes to the user via a user device, where each dish corresponds to a respective recipe in the database. Further, the method also comprises receiving a selection of dishes forming the meal, made by the user. The method additionally comprises determining a required work intensity for/from the user, for each recipe step in a respective recipe associated with the received dish selection, based on information provided by the recipe author. Furthermore, the method comprises determining a completion time interval of each recipe step of the respective recipe, based on information provided by the recipe author. In further addition the method also comprises composing a time interval structure of the various recipe steps of the respective recipe, based on the determined required work intensity of the user for each recipe step and the determined maximum time limit of each recipe step. The method comprises outputting the composed time interval structure of the steps to the user device of the user.", + "p0020": "By using a common recipe format for all recipes which are uploaded and stored on the database, a platform is provided for offering the user a variety of dishes, which may be provided by different recipe authors; which dishes may be combined into a meal, as selected by the user. As the common recipe format requires certain parameters which the recipe author has to specify, such as required work intensity of the user and a completion time interval for each step in the respective recipe, the control unit is enabled to combine the various recipes of distinct dishes into a time interval structure, for assisting and coaching the user when preparing the meal. By preparing the time interval structure of the steps based on the required work intensity of the user and the maximum time limit of each recipe step, it becomes possible for the control unit to optimise the total cooking time and also prepare the meal without any edible becoming deteriorated at the moment of the meal.", + "p0021": "In some embodiments, input values may be collected by the control unit, where after the time interval structure of the recipe steps may be adapted to received input values continuously. The time interval structure may thereby be adapted due to various unexpected events occurring the user during the cooking process, such as received phone calls and various other interruptions.", + "p0022": "Thanks to the provided solution, it becomes possible also for the inexperienced user to compose a complex meal comprising various dishes from distinct recipe authors, also when made with edibles from which the user has none or negligible previous cooking experience. Thereby a tool is provided for the inexperienced user to prepare a meal to a predetermined serving time using an arbitrary selection of recipes.", + "p0023": "In one embodiment of the method of the present invention of cooking a meal, the method comprises the step of providing a cooking device in wireless communication with a control unit. The cooking device is in wireless communication with the control unit. The control unit is in communication with a database, a comparison unit and a communication device. A first cooking recipe is stored in a first format in the database. The first format contains a resource segment, a cooking step or task segment. Each cooking step segment contains a work intensity, a minimum time, a base time and a maximum time. The base time is the same or longer than the minimum time and the same or shorter than the maximum time. The first cooking recipe requires a first resource and at least a first cooking step and a second cooking step. A second cooking recipe is stored in the first format in the database. The second cooking recipe requires a second resource and at least a first cooking step and a second cooking step. The first and second cooking recipes are part of the meal. The communication device sends a request signal to the control unit. The request signal contains a request to plan the meal requiring at least the first cooking recipe and the second cooking recipe and a completion time of the meal. The control unit sends a retrieve signal to the database to retrieve the first cooking recipe and the second cooking recipe. The control unit sends a retrieve signal to the communication device to retrieve a completion time of the meal. The control unit adds up the base times of the first cooking step and the second cooking step of the first cooking recipe to determine a first total time required to complete the first cooking recipe. The control unit adds up the base times of the first cooking step and the second cooking step of the second cooking recipe to determine a second total time required to complete the second cooking recipe. The control unit determines a starting time based on the first total time, the second total time and the completion time. The control unit adds up the work intensity of timewise overlapping cooking steps from the first recipe and the second recipe to a combined work intensity for preparing the meal.", + "p0024": "When the first combined work intensity for the first cooking step of the first recipe and the first cooking step of the second recipe exceeds 100%, the control unit sends a comparison signal to the comparison unit to compare the starting times of the first cooking step of the first recipe with the first cooking step of the second recipe and selects the first cooking step with an earliest starting time and the first cooking step with a latest starting time. The comparison unit sends a control signal to the control unit to advance the completion time of the recipe with the earliest first cooking step and to delay the starting time of the first cooking step with the latest starting time. The control unit iteratively advances the completion time of the first cooking step with the earliest starting time by reducing the base time for this cooking step and increases the base time for the second cooking step of the selected recipe and delays the starting time of the first cooking step with the latest starting time by reducing the base times of this cooking step and the second cooking step for this recipe until the combined work intensity no longer exceeds 100%. When the first combined work intensity for the first cooking step of the first recipe and the first cooking step of the second recipe exceeds 100% and all minimum time limits and all maximum time limits are reached, the control unit sends a message to the communication unit to choose a new combination of recipes for the meal. When the combined work intensity remains at a maximum 100% from the starting time to the completion time, the control unit composes a time interval structure of the steps in the respective recipes based on the optimized cooking schedule to prioritize the cooking steps that have a lowest least work intensity when two or more cooking steps are scheduled to start at the same time. The control unit generates and transmits control signals to the communication device of the user or to a cooking device for outputting the composed time interval structure on the cooking device. The control unit receives at least one input value from a sensor or from the communication device. The control unit composes an adapted time interval structure of the steps with regard to the received input value. The control unit generates and transmits control signals to the communication device and/or to a cooking device for outputting the adapted time interval structure on the communication device and/or the cooking device.", + "p0025": "In an alternative embodiment, the method further comprises the step of the control unit sending a comparison signal to the comparison unit to compare the work intensity of the second cooking step of the first cooking recipe with the first cooking step of the second cooking recipe. The control unit sends an instruction signal to the communication device to start the second cooking step of the first cooking recipe before the first cooking step of the second cooking recipe when the second cooking step of the first cooking recipe has a work intensity lower than the work intensity of the first cooking step of the second cooking recipe.", + "p0026": "In yet another embodiment, the method further comprises the step of providing a sensor on the cooking device. The sensor senses a temperature on the cooking device. The sensor sends a temperature signal to the control unit when the temperature approaches, reach or exceeds a threshold value.", + "p0027": "In another embodiment of the present invention, the method further comprises the step of the control unit receiving the temperature signal at a first time and comparing the first time to a reference time on a countdown timeline, the control unit sending a warning signal to the communication device when the first time is later than the reference time.", + "p0028": "In an alternative embodiment, the method further comprises the step of the control unit receiving the temperature signal at a first time and comparing the first time to a reference time for completing a cooking step and sending a warning signal to the communication device when the first time exceeds a maximum time set for the cooking step.", + "p0029": "In another embodiment, the method further comprises the step of starting a cooking step of the first cooking recipe before a cooking step of the second cooking recipe when the cooking step of the first cooking recipe has a work intensity that is lower than the work intensity of the cooking step of the second cooking recipe.", + "p0030": "In yet another embodiment, the method further comprises the step of the sensor sending information signals directly to the communication device.", + "p0031": "In another embodiment, the method further comprises the step of the control unit comparing resources required for the first cooking recipe with the resources required for the second cooking recipe, further identifying and resolving a conflict in resource utilization.", + "p0032": "In yet another embodiment, the method further comprises the step of the control unit adjusting an permitted preparation time of cooking steps so that the first and second cooking recipes are completed at the same time.", + "p0033": "In another embodiment, the method further comprises the step of the control unit displaying each subsequent cooking step of the first and second cooking recipe along the countdown timeline so that the third cooking step of the first cooking recipe and the third cooking step of the second cooking recipe are simultaneously completed.", + "p0034": "Other objects, advantages and novel features of the described aspects will become apparent from the following detailed description.", + "p0035": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0036": "Various embodiments are described in more detail with reference to attached drawings, illustrating examples in which:", + "p0037": "FIG. 1A is a block diagram illustrating a system according to an embodiment of the invention.", + "p0038": "FIG. 1B is a block diagram illustrating a system according to an embodiment of the invention.", + "p0039": "FIG. 1C is a block diagram illustrating a system according to an embodiment of the invention.", + "p0040": "FIG. 2 is a block diagram illustrating merging of two recipes into a time interval structure of recipe steps.", + "p0041": "FIG. 3 illustrates an embodiment wherein a sensor is measuring a parameter and interacts with the time interval structure of the user.", + "p0042": "FIG. 4 is a flow chart illustrating a method according to an embodiment.", + "p0043": "FIG. 5 is a control unit according to an embodiment.", + "p0044": "FIG. 6 is a schematic illustration of the present invention of two recipes that has a work order that exceeds the 100% maximum work intensity threshold.", + "p0045": "FIG. 7 is a schematic illustration of the present invention of the two recipes shown in FIG. 6 with a revised work order that never exceed the 100% maximum work intensity threshold.", + "p0046": "FIG. 8 is a schematic illustration of system of the present invention showing how signals are sent between components of the system.", + "p0047": "DETAILED DESCRIPTION", + "p0048": "Embodiments of the invention described herein are defined as a control unit and a method, which may be put into practice in the embodiments described below. These embodiments may, however, be exemplified and realised in many different forms and are not to be limited to the examples set forth herein; rather, these illustrative examples of embodiments are provided so that this disclosure will be thorough and complete.", + "p0049": "Still other objects and features may become apparent from the following detailed description, considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the herein disclosed embodiments, for which reference is to be made to the appended claims. Further, the drawings are not necessarily drawn to scale and, unless otherwise indicated, they are merely intended to conceptually illustrate the structures and procedures described herein.", + "p0050": "FIG. 1A is a schematic illustration over a system 100 . The system 100 comprises a control unit 110 for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes. The control unit 110 is connected to a database 120 , wherein recipes 160 , 170 of dishes may be stored. These recipes 160 , 170 may be uploaded to the database 120 over a communication network 115 by various independent recipe authors 130 , 140 . This is possible due to provision of a common recipe format, as will be further discussed in conjunction with presentation of FIG. 2 . The database 120 may store a large number of recipes 160 , 170 .", + "p0051": "As utilised herein; a recipe 160 , 170 comprises instructions how to complete one dish, which consists of one or several work steps in a given work order. A dish is a part of a meal, which dish is corresponding to one recipe 160 , 170 . A meal is a combination of one or several dishes forming the meal. A work step, or recipe step, is an instruction for completing a discrete part of a recipe 160 , 170 . Work order is the order in time in which the work steps are to be completed to successfully cook the recipe 160 , 170 . A time interval is the determined time period for completing each work step where the minimum time limit is given to complete the step and the maximum time limit is given to prevent that the involved edibles become deteriorated. A time interval structure is the combination of time intervals from several recipes 160 , 170 that are to be prepared simultaneously. The common recipe format of the recipes 160 , 170 , is the format in which each work step is described with respect to time, resource and work intensity in such a way that the completion time for each work step may be altered within a given time interval to optimise the total cooking effort when preparing a meal consisting of several dishes that are to be prepared simultaneously to form a complete meal served at a predetermined time.", + "p0052": "A user may via a communication device 150 access the recipes 160 , 170 in the database 120 over the communication network 115 , e.g. presented on a web page or any other similar communication interface. The communication may be wired or wireless in different embodiments.", + "p0053": "The user may be presented various dishes, for example by presenting images on the web page. In some embodiments, recipes 160 , 170 may be searched for in the database 120 on a search engine. Further, recipes 160 , 170 may be searched for on the database 120 e.g. by entering desired ingredients, a particular diet or allergy, etc., in some embodiments. Thereby, the user is enabled to select dishes to be combined to a meal.", + "p0054": "The communication network 115 may comprise a collection of terminal nodes and/or networks, connected wired or wirelessly, so as to enable telecommunication between the terminals. The communication network 115 may thus comprise a computer network or an internet network.", + "p0055": "The control unit 110 may in some embodiments be a server, configured to communicate with the communication device 150 , e.g. over the communication network 115 . The control unit 110 may also in parallel communicate with a plurality of communication devices 150 , depending on capacity of the control unit 110 .", + "p0056": "However, the control unit 110 may in some embodiments not be connected to the communication network 115 , or at least no continuously connected to the communication network 115 . In such embodiments, the communication device 150 may be connected to the control unit 110 , e.g. via a wired connection or via a short range wireless communication such as e.g. Bluetooth, Near Field Communication , Radio-Frequency Identification , Z-wave, ZigBee, IPv6 over Low power Wireless Personal Area Networks , Wireless Highway Addressable Remote Transducer Protocol, Wireless Universal Serial Bus , optical communication such as Infrared Data Association or infrared transmission, etc.", + "p0057": "In some further embodiments, the control unit 110 may be integrated in the communication device 150 of the user. In those embodiments, the control unit 110 may uniquely support the particular user of the communication device 150 in food preparation. Thus the control unit 110 may be a software or an app, which may be downloaded into the communication device 150 , when a connection over the communication network 115 is at least temporarily available and thereafter operate independently of any network connection.", + "p0058": "The communication device 150 of the user in this illustrated embodiment may be represented by a mobile station also known as a mobile device, wireless terminal, mobile telephone, cellular telephone, etc.; a computer tablet, a laptop, a stationary computer, an augmented reality device, a pair of intelligent glasses or lenses, an intelligent watch, a 3D projection device, etc. In some alternative embodiments, the communication device 150 of the user may be integrated into a domestic appliance of the user's kitchen such as a refrigerator, freezer, stove, dishwasher, etc.", + "p0059": "Augmented reality is a live direct or indirect view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video or graphics. It is related to a more general concept called mediated reality, in which a view of reality is modified by a computer. As a result, the technology functions by enhancing one's current perception of the reality.", + "p0060": "With the help of advanced augmented reality technology the information about the surrounding real world of the user becomes visible and possibly interactive. Information about the environment and its objects is overlaid on the real world. This information can be virtual or real, e.g. seeing other real sensed or measured information such as temperature/electromagnetic radio waves overlaid in exact alignment with where they actually are in the kitchen environment, such as the oven, stove, etc.", + "p0061": "The communication device 150 is configured for communication over the communication network 115 .", + "p0062": "The communication device 150 may be configured to receive input from the user via a keyboard, via a touchscreen, via detection of user movements, i.e. a touch-free communication interface, detection and interpretation of voice commands, or similar device for human computer interaction.", + "p0063": "Some examples of such touch-free communication interface may be based on e.g. image recognition or gesture recognition of images captured by a camera comprised in the user's communication device 150 , in some embodiments. In some other embodiments, eye tracking of either the user's point of gaze or the motion of an eye relative to the head, may be applied.", + "p0064": "In some embodiments, a camera and/or movement sensors integrated with the user's communication device 150 , may detect movements of the user in order to enable touch free command input. Voice recognition is another option according to some embodiments.", + "p0065": "An obvious advantage with interaction via a touch-free communication interface is that the user can scroll in the text, swap between different recipes 160 , 170 etc., also when his/her hands are messy. Further, the keyboard/touchscreen of a typical communication device 150 may not be regularly cleaned and therefore comprise germs and bacteria. By avoiding that the user has to physically touch any keyboard/touchscreen of the communication device 150 , it is avoided that bacteria/germs are spread from the communication interface and into the food, which may reduce the risk of food poisoning for the user and his/her meal guests, if any.", + "p0066": "The communication device 150 may output information to the user via presentation of text and/or images on a display, sound emitted via loudspeakers, tactile signals emitted by a haptic device integrated in the communication device 150 , projection of images, etc.", + "p0067": "In some embodiments, the communication device 150 may project a movie and/or an image such as e.g. a 3D image on a wall, fridge or any other similar surface. In some embodiments, a hologram may be created and projected. The movie/image may be outputted by the communication device 150 upon request of the user, or alternatively delivered on command from the control unit 110 when a relevant recipe step is performed.", + "p0068": "In some additional embodiments, the communication device 150 may comprise an augmented reality device which may output various advices and information related to the current recipe step, on command from the control unit 110 .", + "p0069": "FIG. 1B is a schematic illustration over the system 100 illustrated in FIG. 1A , but at a later moment in time where the user has already made a selection of dishes. This selection of dishes is provided to the control unit 110 over the communication network 115 .", + "p0070": "The control unit 110 may then retrieve a respective recipe 160 , 170 , associated with the selected dishes from the database 120 . A time interval structure of recipe steps, such as e.g. a working list to be used by the user is then prepared by the control unit 110 , specifying in which order to perform the different steps of the respective recipes 160 , 170 , and/or the respective completion time of each recipe step. The composition of the time interval structure/work list may be made by the control unit 110 in order to minimise the cooking time, to avoid that food deteriorate by having to wait too long, and also for the dishes to be ready for the meal to be served at approximately the same time.", + "p0071": "The control unit 110 may compose the work list based on required work intensity of the user. Some recipe steps may for example not require any attention at all until the step is about to be ready, but still take some time, such as e.g. defrosting a frozen fish or boiling potatoes. The user may then start performing another recipe step of the other recipe 160 , 170 in parallel, while the fish is defrosting/the potatoes are boiling. Further, some recipe steps may require some surveillance by the user, such as frying sausages in a pan. However, this may not stop the user from being able to simultaneously perform another recipe step, e.g. of another recipe 160 , 170 . There may also be recipe steps which require the full attention of the user, such as for example the mixing of mayonnaise, where inattention of the user is likely to result in wasted ingredients. It is then inappropriate to incite the user to do another recipe step at the same time.", + "p0072": "Further, the control unit 110 may compose the work list by determining which resources are required during each recipe step of the respective recipe 160 , 170 , and prevent any possible clash in required resources by different recipe steps.", + "p0073": "Thereby, upon receiving the work list from the control unit 110 , the user may start preparing the meal step by step by just following the provided work list.", + "p0074": "FIG. 1C is a schematic illustration over the system 100 illustrated in FIG. 1A , but at a later moment in time, alternative to the illustration in FIG. 1B . The user has made a selection of dishes for a meal. This selection of dishes is provided to the control unit 110 over the communication network 115 .", + "p0075": "The control unit 110 may retrieve a respective recipe 160 , 170 , associated with the selected dishes from the database 120 . A time interval structure wherein the user's work of the selected recipes 160 , 170 is then planned and provided to the user's communication device 150 , based on the required work intensity of each recipe step of the respective recipe 160 , 170 , and the maximum time limit of each recipe step. In the illustrated arbitrary non-limiting example, the recipe steps are classified as having 0% intensity; 50% work intensity; or 100% work intensity. The time interval structure may be organised in order to avoid that the user exceeds 100% total work intensity at the same time by adjusting time intervals for the recipe steps, while at the same time avoiding that the maximum time limit for any step is exceeded for avoiding that any food deteriorate by having to wait too long.", + "p0076": "FIG. 2 is a block diagram illustrating merging of two recipes 160 , 170 into a time interval structure of recipe steps/aggregated work list by the control unit 110 , provided to the communication device 150 of the user. This is merely an arbitrary example; another number of recipes 160 , 170 such as e.g. three, four, five, etc. may be combined for a meal in other embodiments.", + "p0077": "The user may select dishes according to the procedure discussed and presented in FIG. 1A .", + "p0078": "A common recipe format is used for all recipes 160 , 170 uploaded to and stored in the database 120 . Different, independent recipe authors 130 , 140 , may be inclined to write their respective recipes 160 , 170 in a different and personal style. It thereby becomes difficult or even impossible to combine recipes 160 , 170 of different recipe authors 130 , 140 into a common work list for preparing a meal. By using the common recipe format and only accept recipes 160 , 170 written on the common recipe format, thereby forcing the recipe authors 130 , 140 to use the common recipe format, it becomes possible to combine different recipes 160 , 170 into a common work list for preparing a meal.", + "p0079": "Thereby, a platform is provided where independent recipe authors 130 , 140 may upload and store recipes on dishes, which then may be displayed and presented to users, thanks to the common recipe format.", + "p0080": "The common recipe format may specify ingredients of the dish. Further the common recipe format may specify a sequence of recipe steps to be performed in order to prepare the dish. Required resources for each recipe step may be specified. It may thereby be avoided that two recipe steps requiring the same resources are scheduled in the common work list at the same time, in some embodiments.", + "p0081": "Further the common recipe format may specify a required user intensity of each recipe step. This may be entered by the recipe author 130 , 140 e.g. as percentages of the user's total amount of attention, in categories, such as \u201cnone\u201d, \u201conly monitoring\u201d, and \u201cfull attention\u201d, as a colour code, as a chart, a diagram staple etc., in some non-limiting examples. It is thereby possible for the control unit 110 to determine how to compose the time interval structure of recipe steps/aggregated work list of the user in an optimal way, so that recipe steps could overlap when none, or only a limited amount of attention is required for at least one of the recipe steps, also when belonging to different recipes 160 , 170 .", + "p0082": "The common recipe format also specifies a maximum time limit for each recipe step of the recipes 160 , 170 . The maximum time limit may be the longest time period which the recipe step could be stalled without deterioration of the ingredients. The maximum time limit of the last recipe step of the recipe 160 , 170 thus indicates how long time the prepared dish could rest before deterioration. The maximum time limit of a stew or pasta sauce may be several hours, or even days while the maximum time limit of pasta may be e.g. half an hour, just to mention some examples. By specifying the maximum time limit of each recipe step, the control unit 110 is enabled to plan and orchestrate the time interval structure, e.g. the order of the work list and time interval length of recipe steps, in order to avoid that any food becomes deteriorated due to having been putted on hold for too long time.", + "p0083": "In some embodiments, the control unit 110 may measure and keep track of the working time of the user of each recipe step. In some embodiments, a warning may be generated and emitted by the control unit 110 to the user's communication device 150 when the maximum time limit of the particular recipe step is approached, reached or exceeded. Thereby, the user may be alerted when the maximum time limit of a particular recipe step is about to be exceeded and may take appropriate measures to avoid that food is deteriorated.", + "p0084": "In some embodiments, an alert may be generated when the user has worked with a recipe step for e.g. 70% of the maximum time, in order to give the user time to finish the critical recipe step. Such alert may be sent as output to the user's communication device 150 e.g., in form of a text message, a visual message, an audio message and/or a tactile message in different embodiments.", + "p0085": "The common recipe format may also specify a minimum time limit for each recipe step of the recipes 160 , 170 , in some embodiments. The minimum time limit may be the briefest possible time period on which the recipe step may be performed. The minimum time limit may for example be less than 1 minute in some embodiments. In other embodiments, for example when boiling pasta, the minimum time limit may be 10 minutes; i.e. it is required to cook pasta for ten minutes in order to be edible, in case the pasta is put into boiling water. The average cooking time will however be longer as it will take some time to boil the water. The minimum time limit of each respective recipe step may be used by the control unit 110 to plan the time interval structure, such as order and time interval length of the recipe steps on the aggregated work list.", + "p0086": "It may be noted that the specified minimum time limit and maximum time limit, respectively, are not related to the actual performance time of the user for each recipe step. Instead the minimum/maximum time limit is/are related to the edible and possible deterioration thereof. It is thereby possible for the control unit 110 to compose the aggregated worklist of the user, focusing on the quality/taste of the edible and the future dish rather than on the capacity of the user, which will result in a more palatable meal.", + "p0087": "By forcing the recipe author 130 , 140 to specify the work intensity and the maximum time limit of each recipe step, and in some embodiments also the minimum time limit of each recipe step and/or the required resources during each recipe step, by the common recipe format, the control unit 110 is enabled to plan the order of the aggregated recipe steps and the completion time of each recipe step on the aggregated worklist in order to optimise the total preparation time of the meal. Optimise in the current sense means avoiding that any food deteriorates and enable serving of all dishes that are supposed to be eaten together in the meal at the same time.", + "p0088": "The work intensity of each recipe step may be indicated to the user, e.g. by a graphic indication or a colour code; where e.g. green may indicate no attention required at all; yellow may indicate some monitoring of the preparation and red may indicate full attention from the user. The work intensity may alternatively be expressed as percentages of the user's total amount of attention, in categories, such as \u201cnone\u201d, \u201conly monitoring\u201d, and \u201cfull attention\u201d, as a chart, a diagram staple etc., as previously mentioned, e.g. in FIG. 1C . It thereby becomes easier for the user to understand the structure of the generated work list and plan the work. An advantage with graphical and/or visual illustration of the work intensity, instead of text based presentation, is that the work intensity is immediately understood by the user, also when he/she is illiterate, dyslectic, stressed or visually impaired.", + "p0089": "Also, in some embodiments, the maximum time limit and/or the minimum time limit of each recipe step may be indicated to the user, not in order to estimate the cooking time, but rather to warn the user that edibles may deteriorate when the maximum time limit for each step is exceeded. Alternatively, the edible may not be ready for consumption when the minimum time limit is not exceeded.", + "p0090": "Furthermore, by indicating the required resources, the user knows what utensils are required so that he/she can avoid dishes requiring utensils he/she does not has access to. The user may also thereby be alerted for re-use of some resources. For example, in case the frying pan is used in two different recipe steps of two different recipes 160 , 170 , the user becomes aware that he/she has to clean the frying pan between the steps or needs to use two frying pans.", + "p0091": "In some embodiments, the time interval structure, or aggregated work list may be continuously monitored and updated by the control unit 110 , e.g. in case the user is/becomes delayed in a recipe step for various reasons. Thus the user in some embodiments may report the termination of each recipe step, and/or the beginning of each recipe step. In other embodiments, the user may inform the control unit 110 concerning an occurred delay, e.g. due to a telephone call or another similar unexpected event. The control unit 110 may then rearrange the time interval structure in order to optimise the food cooking, i.e. avoid that any food deteriorates.", + "p0092": "FIG. 3 illustrates an alternative embodiment wherein a sensor 310 is measuring a parameter and interacts with the time interval structure of recipe steps/aggregated work list of the user, via the control unit 110 and/or the user's communication unit 150 .", + "p0093": "In the illustrated example, the sensor 310 is measuring temperature of a content of a cookware 330 , such as a pot, situated on a kitchen appliance 320 , such as e.g. a stove. The sensor 310 may alternatively measure the inner temperature of an edible, such as a potato, in the cookware 330 . In an illustrative but non-limiting example, the user may be boiling potatoes as a recipe step. The sensor 310 may measure either the temperature of the water in the pot 330 , or the inner temperature of one of the potatoes, in some embodiments. These measurements may be performed by the sensor 310 continuously, or at predetermined or configurable time intervals. The made measurement values may then be provided via wired or wireless signals to the control unit 110 , either directly via a cellular network, WiFi or the like, or indirectly via short range wireless communication such as Bluetooth, Ultra Mobile Broadband, Near Field Communication, Radio-Frequency Identification , Z-wave, ZigBee, IPv6 over Low power Wireless Personal Area Networks , Wireless Highway Addressable Remote Transducer Protocol, Wireless Universal Serial Bus , or similar communication protocol to the communication device 150 of the user, which in turn may forward the made measurement value to the control unit 110 .", + "p0094": "In some embodiments, the temperature of the water in the pot 330 may be measured by the stove 320 , i.e. the sensor 310 may be integrated in the stove 320 , or other kitchen appliances.", + "p0095": "The control unit 110 , may then, upon receiving the sensor measurement values, determine when the cooking step is ready, either by comparing the inner temperature of the edible with a temperature interval associated with the edible, or by computing, based on the measurement values of the sensor 310 and knowledge about cooking time for the particular edible, when the edible may be expected to be ready. The cooking time of e.g. potatoes may be dependent on e.g. if they are put into cold water or boiling water. The cooking time may also be dependent on size of the potatoes, which possibly may be determined by another sensor value, such as e.g. an image taken by the user's communication device 150 in case a camera is comprised therein, or by another sensor 310 of the user, comprising a camera. Or a temperature gauge inserted in the middle of a representative potato in the boiling water. Performance capacity of the stove 320 will also influence the time required for heating a hotplate of the stove 320 . Such information may be obtained by the control unit 110 directly from the stove 320 of the user, in some embodiments.", + "p0096": "When the control unit 110 has estimated that the edible is ready, based e.g. on an input value from the sensor 310 , or a time measurement since cooking begun, an urging may be sent to the user, encouraging him/her to discontinue the cooking/check if the edible is ready, in some embodiments. Such encouragement may comprise e.g. a text message, a visual message, an audio message and/or a tactile message outputted on the user's communication device 150 . In other embodiments, the control unit 110 may generate and send a command to the stove 320 to discontinue the cooking when the edible is estimated to be ready, based on the sensor value/-s.", + "p0097": "The sensor 310 may in some embodiments be integrated in the cookware 330 , kitchen appliance 320 and/or oven or other similar kitchen resource.", + "p0098": "In some embodiments, the sensor 310 may comprise a humidity sensor situated on the surface of the oven for example in some embodiments, alerting the user via an alerting signal generated by the control unit 110 when a humidity exceeding a threshold value is exceeded. In some embodiments, the control unit 110 may generate control signals for decreasing the heat of the stove 320 .", + "p0099": "The above discussed example is merely an example of sensor 310 and how it may be used. The sensor 310 may in some embodiments comprise a thermometer, configured to measure temperature in some embodiments. Such thermometer may comprise e.g. a thermistor, a resistive thermometer, an infrared thermometer, a resistance temperature detector, a heat flux sensor, a pyrometer, a bimetal thermometer, a silicon bandgap temperature sensor, etc., in combination with a wireless transmitter for transmitting the measurement value/-s.", + "p0100": "Such temperature sensor 310 may be applied, besides the example above, for determining temperature of a frozen edible which is to be defrosted ; the temperature of an edible which is prepared in an oven, microwave oven, convection oven, a barbecue, etc.", + "p0101": "However, the sensor 310 may alternatively, or in addition comprise a camera, a stereo camera, an infrared camera, a video camera or similar device, in different embodiments. Thus, photos taken by the sensor 310 may be provided to the control unit 110 . The control unit 110 may upon reception of the images in conjunction with an image recognition program be configured for image recognition/computer vision and object recognition.", + "p0102": "Computer vision is a technical field comprising methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information. A theme in the development of this field has been to duplicate the abilities of human vision by electronically perceiving and understanding an image. Understanding in this context means the transformation of visual images into descriptions of world that can interface with other thought processes and elicit appropriate action. This image understanding can be seen as the disentangling of symbolic information from image data using models constructed with the aid of geometry, physics, statistics, and learning theory. Computer vision may also be described as the enterprise of automating and integrating a wide range of processes and representations for vision perception.", + "p0103": "The image data of the sensor 310 may take many forms, such as e.g. images, video sequences, views from multiple sensors 310 , etc. As already mentioned, the sensor 310 in form of a camera may be comprised in the user's communication device 150 in some embodiments.", + "p0104": "Computer vision may comprise e.g. scene reconstruction, event detection, video tracking, object recognition, object pose estimation, learning, indexing, motion estimation, and image restoration, just to mention some examples.", + "p0105": "According to some embodiments, the control unit 110 may determine that a dish is ready, based on an image captured of the dish, e.g. by analysing the colour of the edible and comparing it with a reference colour scheme, in some embodiments.", + "p0106": "However, in other embodiments, the control unit 110 may determine that an ingredient, such as e.g. a vegetable, is improper or defect, based on an image of the ingredient, captured by the sensor 310 , by applying the image recognition program and comparing the received image/-s with a set of reference images.", + "p0107": "In case a defect ingredient is detected by the control unit 110 , the control unit 110 may generate and transmit an alert to the user. Such alert may be outputted to the user's communication device 150 e.g., in form of a text message, a visual message, an audio message and/or a tactile message in different embodiments.", + "p0108": "In some embodiments, the control unit 110 may as an output control the action of a cooking device of the user, such as the oven, the stove, the micro-oven, the slow-cooker, the rice cooker, etc., based on information received from the user concerning when the meal is to be ready. In some embodiments, the control may be further based on sensor values detected by the sensor 310 , such as e.g. temperature of the food.", + "p0109": "In some embodiments, the sensor 310 may comprise an olfactory sensor, e.g. for determining if an edible is fresh or deteriorated. Based on this information, the control unit 110 may determine if the checked edible is appropriate for human consumption and, if not, trigger the user's communication device 150 to emit a warning to the user, for not using the edible.", + "p0110": "In some embodiments, the sensor 310 may detect bacterial growth on an edible. When this information is received by the control unit 110 , the control unit 110 may compare the detected bacterial growth with a bacterial growth threshold limit. In case the threshold limit is exceeded, the control unit 110 may trigger an alert by the user's communication device 150 that may warn the user from utilising the edible in question.", + "p0111": "FIG. 4 is a flow chart illustrating embodiments of a method 400 in a control unit 110 for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes 160 , 170 stored in a database 120 on a common predetermined recipe format. The recipes 160 , 170 may have been provided by independent, i.e. distinct recipe authors 130 , 140 , thanks to the predetermined common recipe format.", + "p0112": "To appropriately assist the user in meal preparation, the method 400 may comprise a number of actions 401 - 409 . It is however to be noted that any, some or all of the described actions 401 - 409 , may be performed in a somewhat different chronological order than the enumeration indicates. At least some of the actions 401 - 409 may be performed simultaneously or even be performed in an at least partly reversed order according to different embodiments. Further, it is to be noted that some actions may be performed only in some alternative embodiments, such as e.g. actions 405 , and 407 - 408 .", + "p0113": "Action 401 comprises offering a plurality of dishes to the user via a communication device 150 , which dishes correspond to a respective recipe 160 , 170 in the database 120 .", + "p0114": "The dishes may be outputted to the user e.g. in form of images/video sequences of the readymade dishes and/or a descriptive text of the dish. An audio voice presentation of the dishes is another option; or a combination of images, video, text and/or audio presentation.", + "p0115": "Action 402 comprises receiving a selection of dishes forming the meal, selected by the user.", + "p0116": "The user selection of dishes forming the meal the user will prepare may in some embodiments be received over a touch-free communication interface of the user's communication device 150 . Such touch-free communication interface may comprise movement detection and recognition based on camera images captured by the user's communication device 150 of the user. In other embodiments, the touch-free communication interface may be based on voice recognition of audio signals captured by a microphone in the user's communication device 150 .", + "p0117": "Action 403 comprises determining a required work intensity for/from the user, for each step in a respective recipe 160 , 170 associated with the received 402 dish selection, based on information provided by the recipe author 130 , 140 .", + "p0118": "The recipe author 130 , 140 may be forced to specify required work intensity for/from the user for each recipe step, in some embodiments, according to the predetermined common recipe format, when uploading/storing the recipe 160 , 170 in the database 120 .", + "p0119": "Information concerning required work intensity for/from the user, as determined by the recipe author 130 , 140 , may thus be stored in the database 120 , associated with each recipe step.", + "p0120": "Action 404 comprises determining a maximum time limit of each step of the respective recipe 160 , 170 , based on information provided by the recipe author 130 , 140 , according to the common recipe format.", + "p0121": "The recipe author 130 , 140 may be forced to specify the maximum time limit of each recipe step by the predetermined common recipe format, when uploading the recipe 160 , 170 to the database 120 . Thereby, thanks to the common recipe format, and by enforcing the recipe authors 130 , 140 to adapt their recipes 160 , 170 to the common recipe format, it becomes possible to combine recipe steps of different recipes of different recipe authors 130 , 140 into the meal.", + "p0122": "In some embodiments, also a minimum time limit of each recipe step of the respective recipe 160 , 170 , may be determined, based on information provided by the recipe author 130 , 140 . Thereby, in at least some of these embodiments, a time window may be defined for each recipe step, defined by the minimum time limit and the maximum time limit of each recipe step, which time window defines a time period wherein the food prepared during the recipe step is optimal from a taste point of view.", + "p0123": "Action 405 , which may be performed only in some embodiments, comprises determining a resource utilisation of each recipe step of each received 402 selected dish.", + "p0124": "The recipe author 130 , 140 may be forced to specify the resource utilisation of each recipe step by the predetermined common recipe format, when uploading the recipe 160 , 170 to the database 120 .", + "p0125": "Action 406 comprises composing a time interval structure of the various steps of the respective recipes 160 , 170 , such as e.g. a working order of the recipe steps of the respective recipes 160 , 170 , based on the determined 403 required work intensity for/from the user for each recipe step and the determined 404 maximum time limit of each recipe step.", + "p0126": "The time interval structure, such as e.g. an aggregated working order, may also in some embodiments specify a time interval comprising a maximum time limit and possibly also a minimum time limit within which each respective recipe step is to be performed.", + "p0127": "The time interval structure/working order may be composed in order to optimise the meal preparation. By avoiding that food is deteriorated according to the method 400 , it is ensured that the meal will be as tasty as possible.", + "p0128": "The time interval structure/working order may be composed in order to avoid a conflict in resource utilisation between steps of the recipes 160 , 170 , in some embodiments.", + "p0129": "Further, the time interval structure/working order may be composed in order to make the total preparation time of the meal as short as possible in some embodiments. In addition, the time interval structure/working order may be composed in order to get all the dishes ready at the same time, within a time interval, in some embodiments.", + "p0130": "Action 407 , which may be performed only in some embodiments, may comprise receiving at least one input value from a sensor 310 , or the communication device 150 of the user.", + "p0131": "The input value may be e.g. a sensor signal received from a sensor 310 . The sensor 310 may be situated in the kitchen or other similar location of the user. The sensor 310 may be e.g. a temperature sensor, an image sensor, etc., in different embodiments. The sensor 310 may be integrated into a cookware 330 , kitchen appliance 320 or other kitchen resource of the user; or in the user's communication device 150 , etc.", + "p0132": "However, in some embodiments, the input value may be inputted by the user, e.g., upon encouragement of the control unit 110 , for example when a recipe step is terminated or started. The control unit 110 may then activate a time measuring functionality and keep track of the time spent by the user on each recipe step. Thereby, the control unit 110 may predict when the user is approaching the maximum time limit, such as e.g. when it is five minutes' left in a non-limiting example. The user may thereby be encouraged by the control unit 110 to speed up in order to avoid food deterioration, in some embodiments.", + "p0133": "The input value may comprise e.g. a request to stall the food preparation process, e.g. due to a telephone call or other not foreseen event that suddenly occurs while cooking, in some embodiments, which in turn may trigger an adaptation of the time interval structure according to step 408 .", + "p0134": "The user may then stop the process, e.g. by a signal transmitted on the user's communication device 150 .", + "p0135": "Action 408 , which may be performed only in some embodiments, may comprise adapting the time interval structure of the recipe steps with regard to the received 407 input value.", + "p0136": "The adaptation of the time interval structure of the recipe steps may comprise changing an intended time interval of a recipe step, and/or order of recipe steps, etc., however, without exceeding the maximum time limit.", + "p0137": "The adaptation of the time interval structure of the recipe steps may further comprise changing heating temperature of a dish situated in a kitchen appliance 320 and/or a cookware 330 , in order to prolong the cooking time, e.g. when information has been received 407 that the user has been stalled by an incoming phone call or other unplanned event.", + "p0138": "Thus, the time interval structure may be dynamically adjusted to the reality, the capacity of the user and possible occurring interruptions. By not locking the time interval structure to the initially planned time interval structure, the user is helped to handle the cooking, also in a chaotic situation where the user is interrupted by various occurrences.", + "p0139": "Action 409 comprises generating and transmitting control signals to the communication device 150 of the user, for outputting the composed 406 time interval structure of the recipe steps, such as e.g. an aggregated working order.", + "p0140": "The generated and transmitted 409 control signals to the communication device 150 of the user, or to a kitchen appliance 320 of the user, may comprise information concerning the adapted 408 time interval structure.", + "p0141": "The composed 406 time interval structure/working order may then be outputted by the communication device 150 of the user by presenting e.g. a text message, an image message, an audio message comprising voice reading, a projection, an augmented reality image, or a combination thereof in some embodiments.", + "p0142": "Further, the presentation may be outputted on the user's communication device 150 , or possibly in another presentational device situated in the user's kitchen, or a combination thereof.", + "p0143": "In some embodiments, a monitoring of the preparation time of each respective recipe step may be performed continuously by the control unit 110 , and a comparison with the maximum time limit of each recipe step may be made. When the maximum time limit is exceeded, or when the maximum time limit is approached, an alert may be generated and sent to the user, encouraging him/her to speed up and terminate the recipe step. Thereby waste or deterioration of food may be avoided.", + "p0144": "In embodiments wherein the control unit 110 has received 407 an input value from the user, such as e.g. a sensor signal from a sensor 310 associated with the user, the generation and transmission of control signals to the communication device 150 of the user may concern a step of the composed 406 time interval structure of the steps, when the received 407 input value exceeds a threshold limit, associated with the step.", + "p0145": "Some examples may be e.g. a warning signal or voice message when a pan 330 on a stove 320 is about to become too hot. In some embodiments, control signals may instead be sent to the kitchen appliance/stove 320 to adjust the temperature. Some other examples may comprise sending control signals to the communication device 150 of the user to output a warning signal or voice message when it is time to take out a gratin from the oven; or an encouragement to the user to speed up, as an edible in a certain step of the process is about to deteriorate.", + "p0146": "The sensor 310 may be e.g. a temperature sensor, an image sensor, etc. in different embodiments. The sensor 310 may be integrated into a cookware 330 , kitchen appliance 320 or other kitchen resource of the user, in the user's communication device 150 , etc.", + "p0147": "In some embodiments, e.g. during a particular severe part of a recipe 160 , 170 , the generated and transmitted control signals may comprise instructions for the user's communication device 150 to output more detailed instructions, thereby guiding the user through at least a subset of the composed time interval structure of the steps. The outputted information may be e.g. a voice message, a film sequence, an image, a cartoon, a sequence of text messages and images, and/or an augmented reality image stream, etc.", + "p0148": "Furthermore, in some embodiments, the outputted information of the communication device 150 of the user may be an alert or an encouragement to the user to discontinue the recipe step and/or check if the step/edible/dish is ready. The information may be outputted via the user's communication device 150 , e.g. on the display as a visual message such as a text message, as an audio message such as voice reading or a signal, as a tactile signal, etc., or a combination thereof.", + "p0149": "The control unit 110 may in some embodiments generate and transmit command signals for discontinuing a recipe step, e.g. by adjusting temperature of the kitchen appliance 320 , such as a stove, oven, microwave oven, rice cooker, hot pot, etc., without further interaction of the user.", + "p0150": "In some alternative embodiments of the method 400 , the control unit 110 may keep track of each recipe step of the user, e.g. by requesting the user to continuously confirm when starting each recipe step. This may be made continuously through any or all of the recipes 160 , 170 , or only through a subset of the recipes 160 , 170 , such as e.g. a subset which is in particular severe.", + "p0151": "Further, the control unit 110 may compare the time spent on the currently performed recipe step with the maximum time limit associated with the recipe step. When the measured time the user has spent on the recipe step is approaching the maximum time limit, such as e.g. 70%, 80%, 90%, etc., of the maximum time limit. Thereby, the user is alerted when the maximum time limit of a recipe step or resting time of a ready dish is approaching and he/she may speed up the procedure for avoiding that food becomes deteriorated.", + "p0152": "FIG. 5 depicts an embodiment of a system 100 for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes. The system 100 comprises a control unit 110 , configured to perform at least some of the described actions 401 - 409 of the method 400 for assisting a user in orchestrating preparation of a meal comprising a plurality of dishes, based on recipes 160 , 170 stored in a database 120 on a common recipe format, by independent recipe authors 130 , 140 .", + "p0153": "The control unit 110 is configured to offer a plurality of dishes to the user via a communication device 150 , which dishes correspond to a respective recipe 160 , 170 in the database 120 . Further the control unit 110 is configured to receive a selection of dishes forming the meal, selected by the user, via a communication interface. In addition, the control unit 110 is also configured to determine a required work intensity of the user, for each step in a respective recipe 160 , 170 associated with the received dish selection, based on information provided by the recipe author 130 , 140 according to the common recipe format. The control unit 110 is furthermore configured to determine a maximum time limit and, in some alternative embodiments, also a minimum time limit of each step of the respective recipe 160 , 170 , based on information provided by the recipe author 130 , 140 . The control unit 110 is also configured to compose a time interval structure/working order of the recipe steps of the respective recipes 160 , 170 , based on the determined required work intensity of the user for each step and the determined maximum time limit and, in some embodiments, minimum time limit of each recipe step. The control unit 110 is configured to generate and transmit control signals to the communication device 150 of the user, to output the composed time interval structure/working order to the communication device 150 of the user.", + "p0154": "In some embodiments, the control unit 110 may further be configured to determine a resource utilisation of each recipe step of each received selected dish, based on information provided by the recipe author 130 , 140 . The resource may be a cookware 330 , a kitchen appliance 320 , a domestic appliance, a kitchen utensil, etc. Further the control unit 110 may be configured to compose the time interval structure/working order, in order to avoid a conflict in resource utilisation between steps of the respective recipes 160 , 170 , based on the determined resource utilisation of each respective recipe step, in some embodiments.", + "p0155": "The control unit 110 may be configured to generate control signals to output the composed time interval structure/working order to the user by voice reading in some embodiments.", + "p0156": "Further, the control unit 110 may be optionally configured to receive at least one input value associated with the user. The input value may be e.g. a sensor value from a sensor 310 of the user, or the communication device 150 of the user. The control unit 110 may also be configured to generate control signals to output information to the user concerning a recipe step of the composed time interval structure/working order when the received input value, such as e.g. sensor value, exceeds a threshold limit, associated with the recipe step.", + "p0157": "The control unit 110 may in addition also be configured to adapt the time interval structure of the steps with regard to the received input value. Further the control unit 110 may be configured to generate and transmit control signals to the communication device 150 of the user, or to a kitchen appliance 320 of the user, may comprise information concerning the adapted time interval structure, in some embodiments.", + "p0158": "In some alternative embodiments, the control unit 110 may be configured to receive the user selection of dishes forming the meal is received over a touch-free communication interface of the user's communication device 150 .", + "p0159": "The system 100 also comprises a database 120 comprising recipes 160 , 170 , entered on a common recipe format by independent recipe authors 130 , 140 , specifying a required work intensity for/from the user, for each step in a respective recipe 160 , 170 , and a maximum time limit and, in some embodiments, also a minimum time limit of each step of the respective recipe 160 , 170 .", + "p0160": "The database 120 may comprise a Database Management System , i.e. a computer software application that interacts with the user, other applications, and the database 120 itself to capture and analyse data. A general-purpose DBMS is designed to allow the definition, creation, querying, update, and administration of databases. Some arbitrary examples of DBMSs may comprise e.g. MySQL, PostgreSQL, Microsoft SQL Server, Oracle, Sybase, SAP HANA, and/or IBM DB2.", + "p0161": "Further, in some embodiments, the system 100 may also comprise a communication device 150 , of the user. The communication device 150 may typically comprise a mobile cellular telephone, or similar device. The communication device 150 may be arranged, in some embodiments, for touch-free communication with the user.", + "p0162": "Also, the system 100 may further comprise a sensor 310 situated in the user's kitchen or other corresponding location of the user, in some embodiments. The sensor 310 may comprise e.g. a temperature sensor, a camera, a video camera, an infrared camera etc. The sensor 310 may be integrated in a kitchen appliance 320 , or a kitchenware 330 in some embodiments.", + "p0163": "The above described control unit 110 , as illustrated in FIG. 5 may according to some embodiments comprise a receiving circuit 510 configured for receiving uploaded recipes 160 , 170 from recipe authors 130 , 140 . Further, the receiving circuit 510 is configured to receive signals from the user's communication device 150 , e.g. a selection of dishes made by the user. The receiving circuit 510 may be configured to receive signals over a wired and/or wireless communication interface.", + "p0164": "Further, the control unit 110 may according to some embodiments comprise a processor 520 for performing various computations, required for performing the method 400 according to at least some of the previously described steps 401 - 409 . Such processor 520 may comprise one or more instances of a processing circuit, i.e. a Central Processing Unit , a processing unit, a processing circuit, a processor, an Application Specific Integrated Circuit , a microprocessor, or other processing logic that may interpret and execute instructions. The here utilised expression \u201cprocessor\u201d may thus represent a processing circuitry comprising a plurality of processing circuits, such as, e.g., any, some or all of the ones enumerated above.", + "p0165": "Furthermore, the control unit 110 may comprise a memory 525 in some embodiments. The optional memory 525 may comprise a physical device utilised to store data or programs, i.e., sequences of instructions, on a temporary or permanent basis. According to some embodiments, the memory 525 may comprise integrated circuits comprising silicon-based transistors. The memory 525 may comprise e.g. a memory card, a flash memory, a USB memory, a hard disc, or another similar volatile or nonvolatile storage unit for storing data such as e.g. ROM , PROM , EPROM , EEPROM , etc. in different embodiments.", + "p0166": "Further, the control unit 110 may comprise a transmitting circuit 530 in some embodiments. The transmitting circuit 530 may be configured for transmitting a signal to e.g. the database 120 , the user's communication device 150 and possibly other devices associated with the user and/or the user's equipment. The transmitting circuit 530 may be configured to transmit signals over a wired and/or wireless communication interface.", + "p0167": "The previously described actions 401 - 409 to be performed by the control unit 110 may be implemented through the one or more processors 520 within the control unit 110 , together with a computer program for performing at least some of the functions of the actions 401 - 409 . Thus, a computer program, comprising instructions for performing the actions 401 - 409 in the control unit 110 may perform the method 400 according to at least some of the actions 401 - 409 , when the computer program is loaded into the one or more processors 520 of the control unit 110 .", + "p0168": "The computer program mentioned above may be provided for instance in the form of a tangible data carrier carrying computer program code for performing at least some of the actions 401 - 409 according to some embodiments when being loaded into the one or more processors of the control unit 110 . The data carrier may be, e.g., a hard disk, a CD ROM disc, a memory stick, an optical storage device, a magnetic storage device or any other appropriate medium such as a disk or tape that may hold machine readable data in a non-transitory manner. The computer program may furthermore be provided as computer program code on a server and downloaded to the control unit 110 remotely, e.g., over an Internet or an intranet connection.", + "p0169": "As used herein, the term \u201cand/or\u201d comprises any and all combinations of one or more of the associated listed items. The term \u201cor\u201d as used herein, is to be interpreted as a mathematical OR, i.e., as an inclusive disjunction; not as a mathematical exclusive OR , unless expressly stated otherwise. In addition, the singular forms \u201ca\u201d, \u201can\u201d and \u201cthe\u201d are to be interpreted as \u201cat least one\u201d, thus also possibly comprising a plurality of entities of the same kind, unless expressly stated otherwise. It will be further understood that the terms \u201cincludes\u201d, \u201ccomprises\u201d, \u201cincluding\u201d and/or \u201ccomprising\u201d, specifies the presence of stated features, actions, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, actions, integers, steps, operations, elements, components, and/or groups thereof. A single unit such as e.g. a processor may fulfil the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms such as via Internet or other wired or wireless communication system.", + "p0170": "With reference to FIGS. 6 and 7 , one important feature of the present invention is that all the cooking recipes follow the same format that, among other things, includes a time interval to complete each task of the recipes, the required resources and required work intensity for each task of the recipes. Very often a meal requires multiple recipes that are prepared for eating at the same time. An initial step is to determine which of the recipes that takes the longest total time to complete. As described in detail below, each task has a time interval set by the recipe author that contains a minimum time, a base time and a maximum time. The time intervals are used by the recipe author to describe a window in time wherein it is realistic for the user to complete the task and also that the food will keep its good quality. Below the minimum time it is either unrealistic that the user is able to complete the cooking step or that the food stuff will not have time enough to give the desired cooking result. Above the maximum time, the recipe author believes there is a risk that the food may be deteriorated. The base time is set between the minimum and the maximum time and is used to determine the displayed time to complete the cooking step. The base time may vary anywhere from the minimum time to the maximum time. In some embodiments, the base time is allowed to vary within the complete interval range from minimum to maximum time. In other embodiments the base time limits may be adjusted to reflect the user's skill level e.g. so that the minimum time for completing a cooking step is set to some higher value than the minimum time because an inexperienced user might not be able to realistically reach the minimum time in the interval.", + "p0171": "For example, the cooking of spaghetti requires a minimum of 6 minutes of boiling, the base time could be 8 minutes and the maximum time could be 10 minutes. Below the minimum time limit, 6 minutes, the spaghetti will be under cooked and not fit for eating and above the maximum time, 10 minutes in this case, the spaghetti will be overcooked and therefore not at its best for eating. The base time is preferably used to calculate the total time to complete the recipe and also to adjust the cooking order to reach a sustainable work effort for the user when planning from different recipes cooked together to form the complex meal. If the first recipe takes longer to complete, based on the total time base time of all the tasks in the recipe, then this first recipe gets a generally higher priority than the second recipe. Preferably, the first recipe and the second recipe should be completed at about the same time because they are both part of the same meal and should be served together.", + "p0172": "Another important parameter set by the recipe author is the required work intensity to do each task. This is particularly relevant when two recipes must be prepared together at the same time and be completed at the same time to be eaten together as a meal. An important aspect of the present invention is that the method provides guidance to the person simultaneously preparing the two, or more, recipes in which order and how long each task should be performed. The user's/chef's work intensity may be divided into three or more groups such as 5% , 25% or 75% work intensity. A low intensity cooking step or task is where the chef may be passive except for very short moments such as removing a pie from the oven or pouring water over vegetables etc. A medium intensity cooking step or task is where the chef is semi-active during the task and can also perform other tasks during this task. One example of such medium activities is stirring a soup now and then overviewing that a barbeque steak is not too burnt. A high intensity cooking step or task is where the chef is fully active and has difficulties in performing other active tasks during this task, such as chopping vegetables for a salad or whipping cream using a beater.", + "p0173": "It is to be understood that the work intensity of a task may vary between 0% and 100%. It is further to be understood that the person preparing the meal can never perform more than 100% combined work intensity when performing simultaneous cooking steps or tasks. For example, it is not possible for the person to simultaneously do tasks that both require 75% work intensity because the total work intensity would exceed the maximum 100% which defines the user's full work capacity. However, it is thus possible to simultaneously do two tasks wherein, for example, one task requires 75% work intensity and the other 20% work intensity because the total work intensity is 95% which is less than 100%. The percentages are merely illustrative examples and any work intensity percentage from 0 to 100% may be used to define the user's/chef's expected engagement level.", + "p0174": "As best shown in FIG. 6 , a meal plan 598 may be formed by a cooking recipe 600 and cooking recipe 608 in parallel for eating at the same time. This means the recipes should be cooked or be simultaneously prepared so that both recipes are ready to be served at a joint eating time. Recipe 600 requires the completion of a first task 602 , a second task 604 and a third task 606 . Each task is defined by an instruction, a work intensity value and a time interval for completion within which a base time is set to define a realistic completion time for the respective tasks. For example, the first task 602 may have a minimum time of 10 minutes, a base time of 10 minutes and a maximum time of 20 minutes. The second task 604 may, for example, have a minimum time of 20 minutes, a base time of 30 minutes and a maximum time of 40 minutes. Similarly, the third task 606 may have a minimum time of 8 minutes, a base time of 10 minutes and a maximum time of 50 minutes. Similarly, recipe 608 requires the first task 610 , the second task 612 and the third task 614 . For example, the first task 610 may have a minimum time of 2 minutes, a base time of 7 minutes and a maximum time of 10 minutes. The second task 612 may, for example, have a minimum time of 3 minutes, a base time of 21 minutes and a maximum time of 60 minutes. The third task 614 may have a minimum time of 2 minutes, a base time of 12 minutes and a maximum time of 60 minutes. These examples illustrate that the number of tasks and task characteristics vary with each recipe. It is to be understood that a meal plan 598 may comprise more than two recipes and further that the recipes 600 and 608 may comprise more than 3 cooking steps respectively.", + "p0175": "Meal plan 598 includes a countdown timeline 616 in minutes that indicates when a task must be started and ended in order to complete the recipes 600 , 608 at or before the completion or finish time 618 . By adding up the base time of each task of each recipe 600 , 608 the total time required for completing of each recipe can be determined. For example, recipe 600 requires the completion of first task 602 +second task 604 and third task 606 . This totals 50 minutes so recipe 602 must be started at \u221250 minutes before the completion time 618 . Similarly, recipe 608 requires the completion of the first task 610 +second task 612 and the third task 614 . This totals 40 minutes so recipe 608 must be started at \u221240 minutes before the completion time 618 in order to be completed at the same time as the completion of recipe 600 .", + "p0176": "This means the person preparing the meal according to meal plan 598 should start with the first task 602 of recipe 600 . As clearly shown in meal plan 598 , it is not possible to do the first task 602 of recipe 600 and the first task 610 of recipe 608 at the same time because the combined work intensity would be 150% which is higher than 100%. Because there is a conflict of the total work intensity between, for example, first task 602 of recipe 600 with task 610 of recipe 608 , task 610 of the second recipe 608 is delayed to start after the completion of task 602 of recipe 600 , as explained below. The conflict may be characterized by when it occurs in relation to the meal completion time 618 and also how long in time the conflict is. If it turns out that the total minimum time of the subsequent tasks of recipe 608 is too long in order to complete recipe 608 on or before the finish time 618 on the countdown timeline 616 , it is possible to delay the finish time 618 so that the subsequent tasks of recipe 608 can be completed at or before the finish time 618 . In such a case the user is warned that the meal finish time is delayed. It is also possible to extend the subsequent tasks after the identified conflict for recipe 600 so that the start time of recipe step 602 , and therefore also the finish time of recipe step 602 , is advanced so that the conflict is avoided. Another option is to minimize the conflict time by urging the user to speed up the cooking process and thereby complete the conflicting tasks closer to the minimum time of the conflicting cooking steps or tasks. The control unit 110 optimizes the tasks and the order of the tasks for both recipes 600 , 608 so that recipe 608 is completed about the same time as the completion of recipe 600 while maintaining the total work intensity below 100%. The control unit 110 then determines the time flexibility of the subsequent tasks in the recipes to see if the base times can be shortened such that both recipes can be completed at about the same time i.e. so that none of the recipes exceeds the maximum total time that are set to complete both recipes. The total minimum time for the subsequent tasks is compared between the two recipes and the control unit selects 110 the next task for the recipe with the highest total minimum time, as explained below.", + "p0177": "It is also possible to prioritize tasks so that when two tasks are done simultaneously, the task that has the lower work intensity task such as a task that requires 5% work intensity is started just before the start of a high work intensity task such as a task that requires 75% work intensity. In this way, each minute or time interval on the countdown timeline 616 assigns a task to the chef along the time line 616 until both recipes 600 , 608 are completed at the finish time 618 . In operation, a portion of the method of the present invention may be described by the steps below:", + "p0178": "- 1. The control unit 110 retrieves the recipes required for a meal from the database 120 and determines total base time for the required tasks for each of the two recipes i.e. recipe 600 and recipe 608 . This is illustrated in FIG. 6 . - 2. The control unit 110 selects and gives priority to the recipe with the longest base time, such as recipe 600 . - 3. The control unit 110 determine if tasks from recipes 600 and 608 can be done simultaneously by adding the work intensity for each task along a countdown timeline 616 so that both recipes are completed at the desired finish time 618 . This means the control unit 110 starts with the desired finish time 618 and works backward to determine the desired start time 617 . - 4. When the sum of the work intensity exceeds 100% at any period on the countdown timeline 616 there is a conflict because the maximum combined work intensity is set to never exceed 100%. When there is a conflict of the work intensity, the control unit 110 then determines the total minimum time for the remaining tasks for each of recipe 600 and recipe 608 that follows the first task 602 of recipe 600 and the first task 610 of recipe 608 . In other words, the control unit 110 determines the total minimum time by adding up the total minimum times for second task 604 and third task 606 of recipe 600 and by adding up the total minimum times for the second task 612 and the third task 614 of recipe 608 . - 5. In a comparison unit 734 , the total minimum times of the remaining tasks in step 4 are compared to one another when unit 734 received a compare or control signal 736 from control unit 110 to determine which recipe has the longest minimum time of the remaining tasks. The comparison unit 734 sends back a result or control signal 738 to the control unit 110 . Based on the result of the result or control signal 738 , the control unit 110 then selects the next task of the recipe with the highest or longest minimum time after the first task has been completed. In this example the total minimum time for tasks 604 and 606 is 28 minutes while the total minimum time for tasks 612 and 614 is 5 minutes. Because the total minimum time for recipe 600 is greater than the minimum time for recipe 608 , the control unit 110 selects task 604 as the next step when preparing the task order of the meal plan 598 . - 6. The control unit 110 then determines the sum of the total work intensity of task 604 of recipe 600 and task 610 of recipe 608 . In this example, the total work intensity of task 604 and task 610 is 95% which is lower than 100% so step 7a below applies. - 7a. When the total work intensity in step 6 is lower than 100% then the comparison unit 734 compares the work intensity of task 604 with the work intensity of task 610 as required by compare signal 736 and sends back the result signal 738 . Based on the result of the result signal 738 , the control unit 110 gives priority to the task that has the lowest work intensity. In this example, task 604 has a lower work intensity than task 610 so control unit 110 starts task 604 before task 610 . - 7b. When the total work intensity in step 6 is greater than 100% then the control unit 110 determines the total minimum time for the remaining tasks for each of recipe 600 and recipe 608 , as described in step 4. The comparison unit 734 then compares the minimum times for both recipes and sends the report signal 738 back to the control unit 110 . The control unit 110 selects the task of the recipe that has the longest total minimum time of the remaining tasks. In this example, there is no remaining work intensity conflict because the combined work intensity remains at or below 100%", + "p0179": "Steps 4-7 are iterated until all the tasks of recipe 600 and the tasks of recipe 608 have been determined, as best shown in FIG. 7 . In this particular example, it was determined, based on the above method, that the person preparing the meal plan 598 should do the steps in the following order:", + "p0180": "- 1. Recipe 600 , first task 602 ; - 2. Recipe 600 , second task 604 ; - 3. Recipe 608 , first task 610 ; - 4. Recipe 608 , second task 612 ; - 5. Recipe 608 , third task 614 ; and - 6. Recipe 600 , third task 606 .", + "p0181": "The means that the person who is preparing the meal according to the meal plan 598 should follow the above-outlined work order of six steps in chronological order in order to complete both recipes about the same time at the finish time 618 so that the two recipes 600 , 608 can be served together as a meal. The resulting composed time interval structure supplies the user with an optimized step-by-step cooking plan which states that the total work intensity is realistic to perform by a single user and also that the food will maintain the desired quality since the optimized cooking schedule does not exceed 100% combined work intensity and that no time interval is exceeded for any of the recipe cooking steps in the meal plan.", + "p0182": "With reference to FIG. 8 , the control unit 110 sends a retrieve signal 700 to a database 120 to retrieve the first recipe 600 and the second 608 because they are required according to meal plan 598 . It should be understood that the calculations and logic may be performed in the control unit, the database, the user's communication device or any other device. The database 120 sends back a response signal 702 that includes the first recipe 600 and the second recipe 608 . An important feature of the present invention is that all recipes stored in database 120 are stored in a first format 704 . The first format 704 includes segments for the ingredients 706 , the resources required 708 , the cooking steps/tasks 710 wherein each task requires input regarding the work intensity 712 , the minimum time 714 required to complete the task, the base time 716 and the maximum time 718 allowed to complete the task. This means the person creating the recipe is required to provide all the input for each recipe so that all the recipes can be stored in the same format 704 . The first recipe 600 , the second recipe 608 and all other recipes in database 120 are thus stored in format 704 .", + "p0183": "The control unit 110 may be in communication with the cooking device 320 and/or appliance/appliances 330 via a communication signal 720 and antenna 722 , that in turn, is in communication, via a wireless signal 724 , with a sensor 310 that senses the temperature and other parameters of appliance or pan 330 . The sensor 310 may also communicate directly with the communication device 150 via a wireless signal 725 so that the sensor 310 sends information directly to the device 150 and receives information from the device 150 instead of or in addition to the control unit 310 . When the stove 320 is activated an activation signal 726 is sent from the stove 320 via antenna 722 to the control unit 110 . In this way, the control unit 110 may monitor the activity of the stove such as the temperature of the pan 330 . The system may also be automatic so that signals such as starting, finishing and alarm signals are automatically sent to and from the control unit 110 . The activation and de-activation of the stove 320 in activation signal 726 indicates when a task has started and ended. It is also possible to give control signals when a sensor value is approached, reached or exceeded. The control unit 110 can thus monitor the start time and finish time of the various tasks along the timeline 616 . Upon completion of a task, the control unit 110 sends a message 728 via a wireless signal 730 to a display unit 732 which may be part of a communication device 150 such as a smart-phone. The message 728 may include an instruction about what the person preparing the meal according to meal plan 598 should do. This includes instructions about which tasks and in which order the tasks for each recipe should be started and finished.", + "p0184": "Additionally, there is also interaction between the control unit 110 and the communication device 150 in that the user of the device 150 must enter a desired finish time 618 that is sent to the control unit 110 via a return signal 731 . Another example of interaction between the user of the device 150 and the control unit 110 is that the user may indicate the start or completion of a cooking step or task. The control unit 110 then calculates whether it is possible to complete both recipes by the finish time 618 by adding up all the minimum times and/or base times of each task of recipe 600 and separately adding up all the minimum times and/or base times of each task of recipe 608 . A warning signal 730 is sent to the communication device 150 if there is not enough time to complete the recipes before the completion time 618 . When it is possible to complete both recipes on or before the finish time 618 the control unit 110 determines a desirable start time 617 by going through the analysis of steps 1-7 described above. The start time 617 is then sent to the communication device 150 so that the user knows when to start preparing the meal 598 .", + "p0185": "The logic or steps of the present invention may be described or summarized, as shown below:", + "p0186": "- 1. Retrieve recipes from the database; - 2. Summarize the work intensity for all overlapping cooking steps; - 3. Identify time intervals where the work intensity exceeds 100%; - 4. As a first attempt, the control unit identifies the recipe with the cooking step having the last starting time in the conflict and then the control unit determines how much this cooking step can be delayed by using the time buffer between the set base times of the remaining cooking steps in recipe 608 after the conflict and the minimum time for the cooking steps in this recipe; - 5. If the time buffer is larger than the identified conflicting time interval, then the control unit decreases the base time of these cooking steps to resolve the conflict ; - 6. If the time buffer is less than the identified conflicting time interval, then some more time has to be found by adding time to the remaining cooking steps for the other recipe in the conflict and then the control unit increases the base time for these cooking steps of this recipe; - 7. Then the control unit iterates this sequence until there are no longer any conflicting time intervals for overlapping cooking steps having a combined work intensity exceeding 100%. This may be called an optimized cooking schedule; - 8. The optimized cooking schedule is then turned into a composed time interval structure containing a list of cooking steps in a sequence where a cooking step having a low work intensity is prioritized before a cooking step having a high work intensity in the case two cooking steps should be started at the same time; - 9. The composed time interval structure is the sent to the user's communication device and/or to a cooking device; - 10. The method may also include the step of receiving at least one input signal from either a sensor or the user's communication device that cause the control unit to recalculate the composed time interval structure to an adapted time interval structure based on the input signal. There is thus an interactivity between the control unit and the communication device so that the control unit may receive input from sensors in cooking devices and then adapt the time interval structure based on these input signals, and. - 11. Finally, the adapted time interval structure is sent to the user's communication device.", + "p0187": "The optimized cooking schedule may be defined as a state where the control unit has solved all resource conflicts. The composed time interval structure may be defined as a primary listing of cooking steps that is valid until there is a new input from the user and/or a sensor. The adapted time interval structure may be defined as an adapted listing of cooking steps taking the new input into account.", + "p0188": "The described optimization may be performed using iterations, multi variable analysis, convex programming, linear programming, integer programming, quadratic programming, fractional programming, nonlinear programming, stochastic programming, robust programming, heuristics, constraint satisfaction, space mapping, calculus of variation, optimal control, dynamic programming, or any other mathematical model or method.", + "p0189": "While the present invention has been described in accordance with preferred compositions and embodiments, it is to be understood that certain substitutions and alterations may be made thereto without departing from the spirit and scope of the following claims.", + "features": { + "F1": "A method of cooking a meal includes providing a cooking device in wireless communication with a control unit.", + "F2": "The control unit is in communication with a database, a comparison unit, and a communication device.", + "F3": "Storing a first cooking recipe in a first format in the database, the first format contains a resource segment, a cooking step or task segment, each cooking step segment containing a work intensity, a minimum time, a base time, and a maximum time.", + "F4": "The first cooking recipe requires a first resource and at least a first cooking step and a second cooking step.", + "F5": "Storing a second cooking recipe in the first format in the database, the second cooking recipe requires a second resource and at least a first cooking step and a second cooking step, with the first and second cooking recipes being part of a meal.", + "F6": "The communication device sends a request signal to the control unit containing a request to plan the meal requiring at least the first cooking recipe and the second cooking recipe and a completion time of the meal.", + "F7": "The control unit sends a retrieve signal to the database to retrieve the first cooking recipe and the second cooking recipe.", + "F8": "The control unit sends a retrieve signal to the communication device to retrieve a completion time of the meal.", + "F9": "The control unit adds up the base times of the first and second cooking steps of the first cooking recipe to determine a first total time required to complete the first cooking recipe.", + "F10": "The control unit adds up the base times of the first and second cooking steps of the second cooking recipe to determine a second total time required to complete the second cooking recipe.", + "F11": "The control unit determines a starting time based on the first total time, the second total time, and the completion time.", + "F12": "The control unit adds up the work intensity of timewise overlapping cooking steps from the first recipe and the second recipe to get a combined work intensity for preparing the meal.", + "F13": "When the first combined work intensity for the first cooking step of the first recipe and the first cooking step of the second recipe exceeds 100%, the control unit sends a control signal to the comparison unit to compare the starting time of the first cooking steps, selecting the recipe with the earliest starting time.", + "F14": "The comparison unit sends a control signal to the control unit to advance the completion time of the first cooking step for the selected recipe and delay the starting time of the other recipe.", + "F15": "The control unit iteratively advances the completion time of the first cooking step for the selected recipe with the earliest starting time by reducing the base time for this cooking step and increasing the base time for the second cooking step until the combined work intensity no longer exceeds 100%.", + "F16": "When the first combined work intensity exceeds 100% and all minimum and maximum time limits are reached, the control unit sends a message to the communication unit to choose a new combination of recipes.", + "F17": "When the combined work intensity remains at a maximum of 100% from the starting to the completion time, the control unit prioritizes cooking steps with a lowest work intensity when multiple steps are scheduled to start at the same time.", + "F18": "The control unit generates and transmits control signals to the communication device or a cooking device for outputting the composed time interval structure.", + "F19": "The control unit receives at least one input value from a sensor or communication device and generates control signals in response for outputting the adapted time interval structure.", + "F20": "The method includes a step where the control unit sends a comparison signal to the comparison unit to compare the work intensity of the second cooking step of the first recipe with the first cooking step of the second recipe.", + "F21": "The control unit sends an instruction signal to the communication device to start the second cooking step of the first cooking recipe before the first step of the second recipe if its work intensity is lower.", + "F22": "The method includes providing a sensor on the cooking device which senses temperature and sends a temperature signal to the control unit when the temperature approaches, reaches, or exceeds a threshold value.", + "F23": "The control unit receives the temperature signal at a first time and compares it to a reference time on a countdown timeline, sending a warning signal if the first time is later than the reference time.", + "F24": "The control unit receives the temperature signal and compares the first time to a reference time for completing a cooking step, sending a warning if it exceeds the maximum time set for the step.", + "F25": "The control unit may start a cooking step of the first recipe before a step of the second recipe when the step of the first recipe has lower work intensity.", + "F26": "The sensor sends information signals directly to the communication device.", + "F27": "The control unit compares resources required for the first recipe with those for the second, identifying and resolving a conflict in resource utilization.", + "F28": "The control unit adjusts the permitted preparation time of cooking steps so that both recipes are completed at the same time.", + "F29": "The control unit displays each subsequent cooking step of the first and second recipe along the countdown timeline to ensure simultaneous completion of second steps." + } + } + }, + { + "FAN": "78061231", + "CPC": [ + "G06F-003/06/04", + "G06F-003/06/44", + "G06F-003/06/73", + "G06F-005/00", + "G06F-016/00", + "G06F-016/214", + "G06F-016/2219", + "G06F-016/9024", + "G06F-021/60/2", + "G06F-2221/2107" + ], + "Content": { + "title": "Data storage system and method for performing same", + "pa01": "A computer-implemented method for performing data storage. The method comprises the steps of: receiving data to be stored from a data source; segmenting the data into immutable core objects each being written into a collection and being assigned a unique object ID; grouping the immutable objects into blobs each including at least one of the objects and being assigned a unique blob ID derived from the object ID of the at least one of the objects included therein, with the last blob of the collection being identifiable as such; defining a state of the collection by linking at least one subset of the core objects of the collection to one another with a last object of each subset being the object head of the corresponding subset and storing identification information of the head object; and storing each one of the blobs onto at least one backend.", + "c-en-0001": "A computer-implemented method for performing data storage, the method comprising the steps of: receiving data to be stored from a data source; segmenting the data into immutable core objects, each one of the core objects being written into a collection and being assigned a unique object ID; grouping the immutable objects into blobs each including at least one of the objects, each one of the blobs being assigned a unique blob ID derived from the object ID of the at least one of the objects included therein, with the last blob of the collection being identifiable as such; defining a state of the collection by linking at least one subset of the core objects of the collection to one another with a last object of each one of the at least one subset being the object head of the corresponding subset and storing identification information of the head object; and storing each one of the blobs onto at least one backend.", + "c-en-0002": "The method of claim 1 , wherein the step of storing identification information of the head object comprises storing identification information of the head object in the last blob of the collection.", + "c-en-0003": "The method of claim 1 , further comprising the step of selectively retrieving corresponding blobs from one of the at least one backend upon request of the stored data.", + "c-en-0004": "The method of claim 3 , further comprising the steps of encrypting the blobs prior to the step of storing the blobs onto the at least one backend and decrypting the blobs subsequently to the step of retrieving corresponding blobs from one of the at least one backend, the method further comprising the steps of compressing the blobs prior to the step of encrypting the blobs and decompressing the blobs subsequently to the step of decrypting the blobs.", + "c-en-0005": "The method of claim 1 , wherein each one of the core objects has an object type and wherein the step of maintaining the state of the collection by linking at least one subset of the core objects of the collection to one another, with the last object of each one of the at least one subset being the object head, comprises linking active objects of each object type with the last object of each object type being the head object for the corresponding object type.", + "c-en-0006": "The method of claim 1 , wherein the step of storing each one of the blobs onto at least one backend includes storing the blobs onto distributed backends.", + "c-en-0007": "The method of claim 1 , further comprising the step of encrypting the blobs prior to the step of storing the blobs onto the at least one backend and compressing the blobs prior to the step of encrypting the blobs.", + "c-en-0008": "The method of claim 1 , further comprising a step of forking the collection by introducing multiple last blobs being identifiable as the last blob of each forked collection.", + "c-en-0009": "The method of claim 8 , further comprising the step of merging the forked collection into one of a collection and another fork of a collection.", + "c-en-0010": "A computer readable memory having recorded thereon statements and instructions for execution by a computer to carry out the method of claim 1 .", + "c-en-0011": "A data storage system comprising: a core engine module configured to receive data to be stored, store the data onto at least one backend, and retrieve the data from the at least one backend; the core engine module generating and managing at least one collection comprising: core objects each including data to be stored, each one of the objects being immutable and being assigned a unique object ID and wherein at least one subset of the core objects are linked to one another with a last object of each one of the at least one subset being the object head of the corresponding subset; blobs each including at least one of the objects, each one of the blobs being assigned a unique blob ID derived from the object ID of the at least one of the objects included, the last blob of the collection being identifiable as such and including identification information of the head object of each one of the at least one subset; and the at least one of backend storing the blobs thereon and allowing the blobs to be retrieved using the corresponding blob ID.", + "c-en-0012": "The data storage system of claim 11 , wherein the core engine module comprises a core API module configured to open collections and read, write and delete core objects, a collection manager module configured to generate and manage the core objects, and a blob manager module configured to generate and manage the blobs.", + "c-en-0013": "The data storage system of claim 12 , wherein the core engine module further comprises an encryptor module encrypting the blobs prior to storage onto the at least one backends and decrypting the blobs subsequently to retrieval from the at least one backends.", + "c-en-0014": "The data storage system of claim 13 , wherein the core engine module further comprises a compressor module compressing the blobs prior to encryption by the encryptor module and decompressing the blobs subsequently to decryption by the encryptor module.", + "c-en-0015": "The data storage system of claim 11 , wherein link data representative of the links between the core objects of each one of the at least one subset of the core objects is stored in at least a portion of the core objects.", + "c-en-0016": "The data storage system of claim 11 , wherein the unique object ID of each one of the core objects is assigned sequentially.", + "c-en-0017": "The data storage system of claim 16 , wherein the unique blob ID of each one of the blobs corresponds to the object ID of the first core object of the blob.", + "c-en-0018": "The data storage system of claim 11 , wherein each one of the core objects further comprises one of an object alias and a virtual object ID.", + "c-en-0019": "The data storage system of claim 11 , further comprising an index engine performing indexing services to facilitate the organization and location of data stored in a collection.", + "c-en-0020": "The data storage system of claim 11 , wherein the at least one backend of each collection comprises multiple distributed backends to provide distributed storage of the blobs.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a National Stage Application, filed under 35 U.S.C. 371, of International Application No. PCT/CA2017/050708, filed Jun. 9, 2017, which claims the benefit under 35 U.S.C. \u00a7 119 of U.S. provisional patent application No. 62/347,883, which was filed on Jun. 9, 2016. The contents of both are herein incorporated by reference in their entirety.", + "p0003": "TECHNICAL FIELD OF THE INVENTION", + "p0004": "The present invention relates to the field of data storage. More particularly, it relates to a system for performing data storage and to a method for performing the same.", + "p0005": "BACKGROUND", + "p0006": "It is known in the art to use cloud data storage technologies in order to store data on remote, abstract locations, and subsequently retrieve the data, using communication devices connected to a network. In order to proceed to the storage of data, clients typically connect to a cloud storage provider with the communication device, through an application, plugin, API, Web browser, or the like, which stores the data , database records, objects, or the like) on its own servers or datacenters, using known storage technologies to provide resiliency. When using cloud data storage technologies, storage and retrieval of the data is fully handled by the cloud storage provider, such that users are not concerned about material and security aspects of data storage, such as, for example and without being limitative, the hardware equipment, physical location, precise security measures or other requirements of the servers or datacenters where the data is physically stored.", + "p0007": "Known cloud data storage technologies however tend to suffer from several drawbacks. For example, given that client data is stored directly on the infrastructure of the cloud storage provider, the cloud storage provider commonly has access to the data. Hence, clients must trust the security practices, technology and personnel of the cloud service provider with regards to the confidentiality of the stored data. In some cases, it is possible to use encryption to improve security, but even then, the cloud storage provider is generally required to access at least some information. For example, in some instances, the cloud storage provider is required to have access to the encryption key to encrypt and decrypt the data. Even in cases where the provider theoretically never has access to the encryption/decryption keys protecting the client data, it is still required to have access to some unencrypted metadata , which can include important privileged information , to locate and access the data upon request. Therefore, even when encryption is used, the client must have a certain level of trust in the security practices of the cloud storage provider.", + "p0008": "Moreover, given the particularities of each cloud data storage provider, it is not convenient for clients to switch from one cloud data storage provider to another or to distribute their data over multiple cloud storage data providers. For example, in order to distribute data over multiple cloud data storage providers, it is generally required for a client to manually partition the data along some known boundaries and assign specific portions to individual providers. Moreover, if a client subsequently wants to switch between cloud data storage providers , the client is generally required to manually copy the corresponding data from one provider to the other and ensure that all users accessing that data now use the new provider. During such a copy of the data, updates must be suspended or directed to only one of the two providers to ensure integrity. Moving data from one provider to another can also cause failures and loss of metadata as a result of varying services and restrictions of the specific cloud data storage provider.", + "p0009": "One skilled in the art will understand that it is possible to use aggregators to alleviate some of the above-mentioned shortcomings. However, the use of aggregators once again raises confidentiality issues, given that the aggregators generally require access to the data and therefore need to be trusted on the same level as a cloud storage provider would be. In addition, in such a case, performance of the cloud base storage services depends on the aggregator's services and any service failure or performance problem of the aggregator will result in issues for all of the cloud base storage services.", + "p0010": "It will be understood that many of the above-mentioned issues also apply to local data storage or network data storage. For example, when a user wishes to switch from a personal hard disk to another, the user is required to copy all the data from one disk to the other while no updates are being performed. If multiples disks are used, the user is required to manually decide which files or directories should be stored on which disk. In addition, Filesystems support which varies between platforms often can cause failures or result in a loss of metadata when moving/copying data from different filesystems.", + "p0011": "In view of the above, there is a need for an improved system and method for structuring and storing data in a distributed environment, which would be able to overcome or at least minimize some of the above-discussed prior art concerns.", + "p0012": "BRIEF SUMMARY OF THE INVENTION", + "p0013": "According to a first general aspect, there is provided a computer-implemented method for performing data storage. The method comprises the steps of: receiving data to be stored from a data source; segmenting the data into immutable core objects, each one of the core objects being written into a collection and being assigned a unique object ID; grouping the immutable objects into blobs each including at least one of the objects, each one of the blobs being assigned a unique blob ID derived from the object ID of the at least one of the objects included therein, with the last blob of the collection being identifiable as such; defining a state of the collection by linking at least one subset of the core objects of the collection to one another with a last object of each one of the at least one subset being the object head of the corresponding subset and storing identification information of the head object; and storing each one of the blobs onto at least one backend.", + "p0014": "In an embodiment, the step of storing identification information of the head object comprises storing identification information of the head object in the last blob of the collection.", + "p0015": "In an embodiment, the method further comprises the step of selectively retrieving corresponding blobs from one of the at least one backend upon request of the stored data.", + "p0016": "In an embodiment, each one of the core objects has an object type and the step of maintaining the state of the collection by linking at least one subset of the core objects of the collection to one another, with the last object of each one of the at least one subset being the object head, comprises linking active objects of each object type with the last object of each object type being the head object for the corresponding object type.", + "p0017": "In an embodiment, the step of storing each one of the blobs onto at least one backend includes storing the blobs onto distributed backends.", + "p0018": "In an embodiment, the method further comprises the step of encrypting the blobs prior to the step of storing the blobs onto the at least one backend.", + "p0019": "In an embodiment, the method further comprises the steps of encrypting the blobs prior to the step of storing the blobs onto the at least one backend and decrypting the blobs subsequently to the step of retrieving corresponding blobs from one of the at least one backend.", + "p0020": "In an embodiment, the method further comprises the step of compressing the blobs prior to the step of encrypting the blobs.", + "p0021": "In an embodiment, the method further comprises the step of decompressing the blobs subsequently to the step of decrypting the blobs.", + "p0022": "In an embodiment, the method further comprises the step of forking the collection by introducing multiple last blobs being identifiable as the last blob of each forked collection.", + "p0023": "In an embodiment, the method further comprises the step of merging a forked collection into one of a collection and another fork of a collection.", + "p0024": "According to another general aspect, there is also provided, a computer readable memory having recorded thereon statements and instructions for execution by a computer to carry out the above-described method.", + "p0025": "According to another general aspect, there is also provided, a data storage system comprising a core engine module configured to receive data to be stored, store the data onto at least one backend, and retrieve the data from the at least one backend. The core engine module generates and manages at least one collection comprising: core objects each including data to be stored, blobs each including at least one of the objects and the at least one of backend. Each one of the core objects is immutable and is assigned a unique object ID. At least one subset of the core objects are linked to one another with a last object of each one of the at least one subset being the object head of the corresponding subset. Each one of the blobs is assigned a unique blob ID derived from the object ID of the at least one of the objects included. The last blob of the collection is identifiable as such and includes identification information of the head object of each one of the at least one subset. The at least one backend stores the blobs thereon and allows the blobs to be retrieved using the corresponding blob ID.", + "p0026": "In an embodiment, the core engine module comprises a core API module configured to open collections and read, write and delete core objects, a collection manager module configured to generate and manage the core objects, and a blob manager module configured to generate and manage the blobs.", + "p0027": "In an embodiment, the core engine module further comprises an encryptor module encrypting the blobs prior to storage onto the at least one backends and decrypting the blobs subsequently to retrieval from the at least one backends.", + "p0028": "In an embodiment, the core engine module further comprises a compressor module compressing the blobs prior to encryption by the encryptor module and decompressing the blobs subsequently to decryption by the encryptor module.", + "p0029": "In an embodiment, link data representative of the links between the core objects of each one of the at least one subset of the core objects is stored in at least a portion of the core objects.", + "p0030": "In an embodiment, the unique object ID of each one of the core objects is assigned sequentially.", + "p0031": "In an embodiment, the unique blob ID of each one of the blobs corresponds to the object ID of the first core object of the blob.", + "p0032": "In an embodiment, each one of the core objects further comprises one of an object alias and a virtual object ID.", + "p0033": "In an embodiment, the data storage system further comprises an index engine performing indexing services to facilitate the organization and location of data stored in a collection.", + "p0034": "In an embodiment, the at least one backend of each collection comprises multiple distributed backends to provide distributed storage of the blobs.", + "p0035": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0036": "Other objects, advantages and features will become more apparent upon reading the following non-restrictive description of embodiments thereof, given for the purpose of exemplification only, with reference to the accompanying drawings in which:", + "p0037": "FIGS. 1, 1 a and 1 b are flowchart representations of the steps of a computer implemented method for performing data storage, according to an embodiment.", + "p0038": "FIG. 2 is a schematic representation of an embodiment of an object used for data storage according to the method of FIG. 1 .", + "p0039": "FIG. 3 is a schematic representation of an embodiment of a blob used for data storage according to the method of FIG. 1 .", + "p0040": "FIG. 4 is a schematic representation of an exemplary embodiment of a collection including a plurality of blobs each having a blob ID corresponding to the object ID of a first object of the blob, with the objects being numbered sequentially.", + "p0041": "FIG. 5 is a schematic representation of an exemplary embodiment of the storage of a file using the method of FIG. 1 .", + "p0042": "FIG. 6 is a schematic representation of an exemplary embodiment of a collection management using last object types and object linking and the impact on garbage collection.", + "p0043": "FIGS. 7 a , 7 b , 7 c and 7 d are schematic representations of exemplary embodiments of forking of a collection.", + "p0044": "FIG. 8 is a schematic representation of a system for performing data storage, in accordance with an embodiment.", + "p0045": "FIG. 9 is a schematic representation of the system for performing data storage of FIG. 8 , in accordance with an embodiment where an application uses a filesystem engine module and/or a database engine module for communicating with the core engine module.", + "p0046": "DETAILED DESCRIPTION", + "p0047": "In the following description, the same numerical references refer to similar elements. Moreover, although the embodiments of the system and method for distributed storage of data consist of certain components as explained and illustrated herein, not all of these components are essential and thus should not be taken in their restrictive sense. It is to be understood, as also apparent to a person skilled in the art that other suitable components and cooperation thereinbetween may be used for the system and method for distributed storage of data, as will be briefly explained herein and as can be easily inferred herefrom by a person skilled in the art.", + "p0048": "In general terms, the present system and method for storage of data relies on small, discrete, independent storage units referred herein as \u201ccore objects\u201d, which are agglomerated into larger entities referred herein as \u201cblobs\u201d, with the blobs being stored in an open-ended, log-like storage structure under which different filesystems, databases, or other types of data can be stored and distributed over multiple, disparate and/or otherwise potentially incompatible storage mediums. In an embodiment, in order to increase security, the blobs are compressed and encrypted before being stored and decrypted and decompressed when retrieved. In an alternative embodiment, the core objects can also be individually compressed and encrypted within the corresponding blob , in addition to the blobs being themselves compressed/decompressed and encrypted/decrypted. One skilled in the art will understand that other terms, such as \u201cpods\u201d or the like could have been used to refer to the storage units referred herein as blobs.", + "p0049": "As will become apparent in view of the description below, the system and method for storage of data allow the blobs to be freely moved, copied and shared between various and otherwise potentially incompatible storage providers and/or devices, without requiring central coordination, tracking or synchronization and even while concurrent updates are in progress.", + "p0050": "In the course of the present document, the term \u201ccore object\u201d is used to refer to basic storage units which can include any arbitrary logical data, such as, for example and without being limitative database records, file fragments, metadata, directory indexes, or the like. The term \u201cblob\u201d is used to refer to the containers including multiple core objects and which are stored on the storage vessels known as \u201cbackends\u201d . The term \u201cbackend\u201d is used to refer to autonomous storage vessels and includes for example and without being limitative, local hard disks, flash memory, cloud storage providers, and zones, buckets or other segregation units thereof, network attached drives, or any storage media or service capable of storing data. The term \u201ccollection\u201d is used to refer to a group of core objects, blobs and backends forming a coherent unit that can be managed as a whole. A collection can represent one or more user-visible filesystems, databases, object stores, or any other form of grouping needed to keep related data together. Other designative terms such as \u201ccosmos\u201d could have been used to refer to a group of core objects, blobs and backends forming a coherent unit.", + "p0051": "Referring generally to FIG. 1 , there is provided an embodiment of a method for performing data storage 10 , which allows data to be freely distributed and hosted on various devices and services with high flexibility and security. In the embodiment shown, the method for performing data storage 10 includes the steps of receiving data to be stored from a data source , segmenting the data into core objects , grouping the core objects into blobs , compressing the blobs , encrypting the blobs , storing the blobs on at least one backend and selectively retrieving corresponding blobs from one of the at least one backend upon request of the stored data .", + "p0052": "Referring to FIGS. 1 and 2 , as mentioned above, the data held into the core objects 50 generated at step 12 can be any form of arbitrary, binary data. In an embodiment, each core object 50 includes a header 52 describing the contents of the core object 50 and a payload 54 containing the actual data for the core object. For example and without being limitative, the header 52 can include information such as, Object ID 52 a , Object Type 52 b , Virtual ID 52 c , name 52 d , Universal Unique ID 52 e , flag indicating whether the object is being tracked as the last object of its type 52 f , links to related objects 52 g , ID of object replaced by current object 52 h and/or ID of other objects deleted by current object 52 i . As will be described in more details below, the OID 52 a is an identifier of the object and the OType identifies the purpose and format of the content of the core object 50 , which helps in decoding the payload. The Virtual ID 52 c , name 52 d , Universally Unique ID 52 e , or other types of identifiers, can be referred to as object identification information which allows objects 50 to be located using alternate identifiers other than the official OID 52 a . The flag indicating whether the object is being tracked as the last object of its type 52 f , links to related objects 52 g , ID of object replaced by current object 52 h and/or ID of other objects deleted by current object 52 i is collection management information which can be used for collection management, as will be described in more details below. One skilled in the art will however understand that, in an alternative embodiment, the core objects 50 can be structured differently and/or contain different information than in the embodiment shown.", + "p0053": "Every core object 50 generated is immutable, i.e. the content of the core object is permanent and cannot be altered once the object has been created, committed and written to a collection. In view of the above, the content of an object cannot be updated. Therefore, in the case where the content of an object needs to be updated, a new core object 50 must be created. In an embodiment, the old object can be flagged as \u201creplaced\u201d so it may be subsequently removed from the collection, as will be described in more details below with regards to garbage collection. One skilled in the art will also understand that, in an embodiment, instead of replacing an entire object, the content of an updated core object may also be reflected by creating a second core object containing only differential information about the original core object.", + "p0054": "Moreover, each one of the core objects 50 is identifiable by the primary identifier referred to as the Object ID 52 a . The OID 52 a is a collection-unique, immutable, arbitrary-length identifier assigned to the core object 50 when it is created. The collection-unique character of the OID 52 a means that each OID 52 a is assigned only once to a given core object 50 , and is never reused for the lifetime of the collection, even after a core object 50 is deleted. When combined with immutability, the collection-unique OID 52 a becomes a de facto signature of a given immutable core object 50 . In other words, the collection-unique character of the OID 52 a creates a permanent and inseparable relationship between a given OID 52 a and the precise, invariable contents of the core object 50 . Given that core objects 50 are immutable and OIDs 52 a are collection-unique, objects cannot be updated, when data associated to a specific core object 50 is updated, at least one new core object 50 associated to the updated data is created and is assigned its own new unique OID 52 a . The collection management for managing data update will be described in more details below.", + "p0055": "In an embodiment, the collection-unique OIDs 52 a are assigned to the generated core objects 50 using ever-increasing arbitrary-length sequential integers as OIDs 52 a , i.e. each newly created core object 50 is assigned an OID 52 a having an integer value corresponding to the last attributed OID+1. One skilled in the art will however understand that, in alternative embodiments, other OID assignation practices can be used for assigning a collection-unique OID to each newly generated OID.", + "p0056": "Referring to FIGS. 1, 3 and 4 , in an embodiment the core objects 50 are assembled at step 14 into a raw blob 60 a , which corresponds to the raw content of the blob of FIG. 3 , before compression and encryption is applied. As mentioned above, in an embodiment , the core objects 50 can be individually compressed and encrypted. In the embodiment shown, the raw blob 60 a includes a stream of all core objects 50 contained in the blob 60 . In an embodiment, the raw blob 60 a also includes information about the last core objects 50 of the collection when the blob 60 was written . For example and without being limitative, such information can include a list of core object types 52 b and OIDs 52 a for objects that are tagged as the last core objects 50 of their types when the blob 60 was written . Once again, the information about the last objects 50 of the collection when the blob 60 was written is collection management information which is used for collection management and will once again be described in more details below.", + "p0057": "In an embodiment, the raw blob 60 a is compressed into a compressed blob 60 b and the compressed blob 60 b is encrypted , thereby resulting in the encrypted blob 60 c . One skilled in the art will understand that, in an embodiment, the raw blob 60 a or compressed blob 60 b could be directly used as blob 60 , without compression/encryption, compression/encryption being used to reduce the size of the blobs and increasing security thereof during storage.", + "p0058": "In view of the above, in an embodiment where the raw blobs 60 a are encrypted, the blobs 60 contain both data and metadata information and are identified solely by their BID. Consequently, the data and metadata is fully encrypted before it is stored on the backend . In an embodiment, each blob 60 is individually tagged as to the encryption algorithm used, which allows the blobs 60 to be individually and autonomously encrypted using various encryption algorithms and keys. Such individual encryption increases the confidentiality of the data stored using the present method for performing data storage.", + "p0059": "One skilled in the art will understand that the directory, file, or other data to be stored in the core objects 50 can themselves be individually encrypted with private encryption keys, before being stored using the present method, such that data can be shared securely with other users. In other words, even users which have access to blob decryption keys to read a collection can be prevented from reading the data itself unless the user has the appropriate data decryption keys. This effectively allows a collection to be shared with multiple users, with full security, without relying on central servers or software-based protections.", + "p0060": "In an embodiment, all blobs 60 include an initial chain checksum 61 c , which is a checksum value that is calculated from the blob 60 when it was created, plus the chain checksum of the previous blob 60 . This chain checksum 61 c creates a unique chain value of all blobs that were ever created in the collection. For example and without being limitative, in operation, the verification that the last blobs of all backends have matching chain checksums 61 c can ensure that all the blobs 60 belong to the same collection. All blobs 60 can also include a Keyed hash message authentication code 61 e designed to cryptographically confirm that the blob 60 was produced by an authorized user and has not been forged and an integrity checksum value 61 b that helps to quickly detect errors that may have been introduced during the transmission or storage of the blob 60 .", + "p0061": "In an embodiment, the blobs 60 can also include a block chain validation hash structure 61 d which can be used to validate a chain of blobs of a collection, by a peer network, in accordance to known block chain validation principles, methods and process. For example, in an embodiment, block chain validation can be performed without transmitting or disclosing the content of the blobs, by having the network validate the authentication digest of the blob instead of the blobs themselves. It will be understood that, if the authentication digest is used for block chain validation as described above, the payload hashed by the authentication digest should also include the digest of the previous blob.", + "p0062": "Each one of the blobs 60 is identified by a collection-unique Blob ID 61 a . In order to allow identification of the blob 60 that contains a given object 50 directly from the OID 52 a of a sought core object 50 , without using indices or maps, the collection-unique BID 61 a is a derivative of the OIDs 52 a of the objects 50 contained in the specific blob 60 . For example and without being limitative, in an embodiment the ever-increasing arbitrary-length sequential integer attributed OID 52 a of the first object 50 in the blob 60 is used as BID 61 a for the blob 60 . Using this implementation, the blob 60 containing a specific object 50 can easily be identified as the blob 60 with the highest BID 61 a that is lower or equal to the sought OID. One skilled in the art will understand that, in alternative embodiments other implementation where the collection-unique BID 61 a is a derivative of the OIDs 52 a of the objects 50 contained in the blob 60 can also be used. Similarly to OIDs 52 a , BIDs 61 a are generally also assigned only once to a given blob 60 , and remain unique for the lifetime of the collection. In an embodiment, the BIDs 61 a can be encrypted, to avoid disclosing the actual sequence of blobs.", + "p0063": "In an embodiment, BIDs 61 a are also predictable, such that it is possible to predict the next BID 61 a that will be written in a collection and to identify the last blob of a collection. In the embodiment where the ever-increasing arbitrary-length sequential integer attributed OID 52 a of the first object 50 in the blob 60 is used as BID 61 a for the blob 60 , the BIDs 61 a are implicitly predictable. In FIG. 4 , the four blobs 60 are provided with blob having BID # 1 including objects having OID # 1 to # 13 , blob having BID # 14 including objects having OID # 14 to # 44 , blob having BID # 45 including objects having OID # 45 to # 46 and blob having BID # 47 including objects having OID # 47 to # 63 . In such an embodiment, the next BID corresponds to the OID of the last object 50 of the last blob 60 plus 1 and the last blob of a collection is simply the blob 60 with the highest BID. Once again, one skilled in the art will understand that, in alternative embodiments, alternative implementations for producing predictable BIDs and identifiable last blob of a collection can also be used.", + "p0064": "In an embodiment, the generated blobs 60 can be subsequently split, with the split blobs being assigned new collection unique BIDs 61 a which are each a derivative of the OIDs 52 a of the core objects 50 now contained the specific blob 60 . It will be understood that, when blobs are split, the assigned BIDs 61 a must still allow identification of the blob 60 that contains a given object 50 directly from the OID 52 a of a sought core object 50 . For example, in the embodiment where ever-increasing arbitrary-length sequential integer attributed OIDs of the first object 50 in the blob 60 is used as BID 61 a for the blob 60 , a split should always result in core objects 50 with adjacent OIDs being grouped in corresponding blobs 60 , with the OID 52 a of the core object 50 having the lowest OID 52 a being used as BID 61 a for each new blob 60 .", + "p0065": "Moreover, in an embodiment, adjacent blobs 60 can also be fused together, with the fused blob being assigned a new collection unique BID 61 a which is a derivative of the OIDs 52 a of the core objects 50 now contained the specific blob 60 . Once again, the assigned BIDs 61 a must still allow identification of the blob 60 that contains a given object 50 directly from the OID 52 a of a sought core object 50 . For example, in the embodiment where ever-increasing arbitrary-length sequential integer attributed OIDs of the first object 50 in the blob 60 is used as BID 61 a for the blob 60 , a fusion should always result in the OID 52 a of the core object 50 having the lowest OID 52 a being used as BID 61 a for the new blob 60 resulting from the fusion.", + "p0066": "In an embodiment, the core objects 50 of a given type 52 b can be grouped in separate blobs 60 from core objects 50 of a different Otype 52 b . Hence, in such an embodiment, the blobs 60 are also associated to a type and the blobs 60 of different types can be stored on different backends 70 . This can be used for example to isolate metadata from data objects in order to improve metadata read performance, to implement different object encryption algorithms/keys based on the Otype 52 b , etc. In such an embodiment, in order to support typed cores objects 50 , blobs 60 and backends 70 , the BID 61 a of a given blob 60 can be derived from the OID 52 a of the corresponding core objects 50 at the backend level rather than at the collection level, with the blobs 60 storing core objects 50 having discontinuous OIDs 52 a . In other words, the BID 61 a of a given blob 60 can be derived from the OID 52 a of the corresponding core objects of the specific Otype 52 b rather than from the OID 52 a of all generated core objects, thereby resulting in core objects 50 having discontinuous OIDs 52 a being stored in the blobs 60 . Therefore, any subsequent request to retrieve a core object 50 should be handled taking into account the Otype 52 b of core object 50 to be retrieved, for example by querying only the backend 70 associated to the specific Otype 52 b of the core object 50 to be retrieved.", + "p0067": "In another embodiment, the blobs 60 can also be isolated based on their object types 52 b , such that certain Otypes 52 b are preferably not written in the same blob. For example, large data objects can be requested to be isolated from metadata and index objects, thereby improving performance by preventing large amounts of data to be retrieved at the same time metadata or index information is retrieved. Hence, in an embodiment, before performing assembly of core objects 50 into a blob 60 the Otype 52 b of the core objects 50 to be assembled into the blob 60 can be considered to determine whether an object should be added to an existing blob 60 or created in a new blob 60 .", + "p0068": "Now referring to FIG. 5 , an example of storage of a file 30 using the above described structure for the method of storing data 10 is shown. In the embodiment shown in FIG. 5 , a file 30 is stored in the cloud using the above described method 10 . In the embodiment shown, the file 30 is segmented into three objects 50 , one containing metadata such as the filename, creation date, etc., and two containing the content of the file as data fragments. The first two objects 50 are assembled into a first blob 60 , and the third object is placed in another blob 60 . The blobs 60 are compressed, encrypted, and transmitted through a secure connection on the Internet to a cloud storage provider acting as backend 70 , where the blobs 60 can be stored as files, database records, objects, or raw storage. In the embodiment shown, a copy of the blobs 60 is also kept in a local cache , which can also be used as direct storage medium. It will be understood that in order to retrieve the stored data, any copies of the blobs 60 can be used to perform the reversed process where the blobs 60 are read, decrypted, decompressed, and the file is reassembled from its segments.", + "p0069": "In view of the above, it will be understood that at step 20 , the step of storing of the blobs 60 on at least one backend is performed to store the blobs onto storage mediums. As mentioned above, the backends 70 are autonomous storage vessels and are simply required to be able to query, store and retrieve blobs using their BID 61 a , in the form appropriate for the supported service or device . Backends 70 are not required to support metadata, structures, know how objects or blobs are organized, their purpose or the like. Backends 70 are, by nature, fully generic, simple storage appendages that blindly store and retrieve blobs 60 based on their BIDs 61 a . In the embodiment shown, the backends 70 are embodied by a cloud storage provider and the local cache, but one skilled in the art will understand that in alternative embodiments, the backends 70 can be any storage media or service capable of storing data. In view of the above described structure, differences in backend storage's abilities and features have substantially no impact. It will be understood that multiple distributed backends 70 can be used such that distributed data storage can be performed by storing the blobs into different distributed backends 70 . Moreover, the blobs 60 can be freely scattered over multiple backends 70 and subsequently moved between backends 70 without any impact on data integrity.", + "p0070": "As mentioned above, the blobs 60 , core objects 50 and backends 70 are grouped together in a coherent unit that can be managed as a whole and referred herein as the collection 80 . Among other things, the collection 80 ensures that OID's 52 a and BID's 61 a are always unique for the collection 80 . In an embodiment, for each collection 80 , the collection management information stored in the core objects 50 and blobs 60 and defined above is used as primary mechanism to hold the entire collection coherent and up to date.", + "p0071": "Referring to FIGS. 1 to 4 , in an embodiment, the method 10 includes a further step of maintaining a collection state 22 reflecting the state of the collection at a specific time where a blob 60 is written. In operation, in order to allow the maintaining of the collection state, the collection management information of each blob 60 includes the state of the collection and its core objects 50 when the blob 60 was written. In an embodiment, the state of the collection includes the list of object types 52 b and OIDs 52 a for last core objects 50 of each Otype 52 b that are tagged as the last objects when the blob 60 is written . Indeed, as mentioned above, in an embodiment, core objects 50 have a given type 52 b designed to indicate the general nature of each core object 50 . When a core object 50 is created, it can be flagged as the \u201clast\u201d core object of its Otype 52 b . In an embodiment, the OID 52 a of each last core object 50 for each Otype 52 b is recorded in the state of the collection of the last blob 60 , such that the last core object 50 of a given Otype 52 b can subsequently be retrieved, without having to know its OID 52 a . Evidently, in order to locate the last core object 50 of a given Otype 52 b , identification of the last blob 60 of a collection is also required.", + "p0072": "Moreover, the collection management information includes object linking information allowing core objects 50 to refer to one another. The object linking information of a particular core object 50 is stored directly in the corresponding core object 50 and defines the bulk of the structure by which the core objects 50 , the blobs 60 and the collection 80 are held together in a tree-like structure of core objects 50 linked to one another. In an embodiment, the collection 80 is defined by \u201chead\u201d objects 50 , which corresponds to the core objects 50 being flagged as the last of each object types, and then linked successively to other core objects 50 , grouping objects 50 as a tree-like chain of core objects 50 . In such an embodiment, the \u201ctail\u201d of the collection 80 acts as the \u201chead\u201d of all objects 50 within the collection 80 . It will be understood that in an alternative embodiment, alternate ways of identifying and linking objects can also be used. Moreover, in an alternative embodiment, subsets different than core object types 52 b can be used to link core objects 50 to one another with the last object being the head object.", + "p0073": "In the case of a data update , the links between core objects 50 need to be updated in order to reflect that the new core object 50 now replaces the old core object 50 . In other words, the core objects 50 previously linking to the old core object 50 by OID 52 a now need to link to the new core object 50 by OID 52 a . Depending on the complexity of the tree structure of the objects 50 , this could lead to a long cascade of referring core objects 50 that successively need to be updated. In order to reduce the required cascade effect, core objects 50 can be structured according to known data structure limiting such cascading, such as for example and without being limitative, by including within the updated core objects 50 information about the OID link updates that would need to be applied to the parent object, without actually updating the parent object itself, or by using non-OID links as described below as an alternative embodiment, at strategic points in the object trees in order to prevent OID link updates from cascading to the top of the tree. In alternative embodiments, object identification information such as, aliases, virtual IDs, private indices or the like, which are alternate identifiers by which an object may also be referred to, in addition to its OID 52 a and which can be reassigned later on to newer core objects 50 can also be used. For example and without being limitative, in an embodiment, each core object 50 can include a Virtual ID 52 c which is initially identical to the OID 52 a . When a new core object 50 is created and is flagged as replacing an older core object 50 , the new core object 50 can automatically inherit the VID 52 c of the older core object 50 . Internally, VIDs 52 c can be stored in VID/OID pairs to allow VID/OID association. Hence, VIDs 52 c can be used to link core objects 50 together using an identifier which is automatically kept up to date when objects themselves are updated and replaced. In another alternative embodiment, private indices can be used to track and link objects 50 externally by their OID 52 a , instead of storing links inside objects.", + "p0074": "In an embodiment, the method for performing data storage 10 can further include a step of performing garbage collection 24 , to reclaim storage space occupied by core objects 50 which are no longer valid, either because they have been replaced by newer core objects 50 , deleted, or because they are no longer being referenced by any other core object 50 . In an embodiment, the step of performing garbage collection 24 includes a step of identifying objects to be cleaned-up 24 a , which can be implicit or explicit and a step of physically removing objects from the collection and blobs 24 b.", + "p0075": "In an embodiment, during the step of flagging 24 a , core objects 50 can be implicitly flagged as no longer required when they lose all references from other core objects 50 . One skilled in the art will understand that several methods for flagging objects as no longer being referenced can be used. For example and without being limitative, in an embodiment, core objects 50 being flagged as unreferenced can be added to a purge candidate list being processed during the purging step 24 b , with the last core objects 50 written in a collection never being added to the purge candidate list even if they are unreferenced in order to protect collections that are being actively updated. Moreover, in an alternative embodiment, core objects 50 can also be explicitly flagged as eligible for garbage collection, either by being flagged as \u201creplaced\u201d when new core objects 50 are created to replace them, or as \u201cdeleted\u201d by core objects 50 that explicitly delete them. In such cases, the replaced/deleted core objects 50 OIDs 52 a can be added to an internal index, along with the OIDs 52 a of the objects that replaced/deleted them, with the index being used as the purge candidate list.", + "p0076": "During the purging step 24 b , the core objects 50 and blobs 60 that are listed in the purge candidate list are physically removed. One skilled in the art will understand that various purge parameters, such as blob usage thresholds, minimum age of objects that should be purged, or the like, can be set to optimize the purge process. The purge parameters are useful in order to limit the purge of objects, for example and without being limitative for archiving purposes, as will be described in more details below. It will be understood that blobs 60 can be purged fully or partially. In a case where all objects 50 within a blob 60 are to be physically removed, the blob 60 can be completely deleted from the backend 70 . However, in a case where only some objects 50 of a blob 60 are to be physically removed, the blob 60 can be opened , the specific objects 50 can be removed, and the blob 60 can be recompressed, re-encrypted, and written back to the corresponding backend 70 under the same BID 61 a.", + "p0077": "One skilled in the art will understand that garbage collection is not required to be identical for each backend 70 . In other words, partially or fully purged blobs do not need to be synchronized across backends 70 . Some backends 70 can continue to host blobs 60 with all of the original objects 50 in them while other backends 70 can have only cleaned-up blobs 60 , without affecting the integrity of the collection 80 . One skilled in the art will understand that, since all blob versions contain the exact same copy of active immutable objects 50 , any version of a blob 60 can be used to reflect the current state of the collection 80 . Variations between versions of blobs 60 only affect the amount of storage used, and the availability of historical data . One skilled in the art will understand that, this asymmetry can be useful for archival purposes.", + "p0078": "For example and without being limitative, in an embodiment, high-performance backends 70 such as local hard disks, SSD drives, SAN and cached cloud storage or the like can be designated to host fully purged blobs 60 that contain only the most recent objects 50 , while other lower-performance or lower-cost backends 70 such as WORM devices , low cost & performance hard disks, and long-term archival cloud storage services, or the like can be designated to host blobs with a greater quantity of original objects 50 .", + "p0079": "Now referring to FIG. 6 , an example of collection management using last objects types, linking and their impact on garbage collection in accordance with an embodiment is shown. In the example of FIG. 6 , the evolution of the collection 80 is followed blob 60 by blob 60 .", + "p0080": "Initially, at step 1 , two core objects 50 are written in Blob # 1 . At this initial stage, neither core object 50 is tracked as the last object of its type 52 b . The core objects 50 do not contain links and are not linked to. At this point, Object # 1 and Object # 2 are considered unreferenced objects, but are not considered as purgeable core objects 50 as they are the very last objects of the collection.", + "p0081": "At step 2 , two more core objects 50 are added and are written in Blob # 3 . Core object # 4 is tracked as the last core object 50 of its type 52 b , and contains links to core objects # 1 , # 2 and # 3 , thereby generating a tree of objects, with core object # 4 as the head, and linking all other objects, including those written in Blob # 1 . The head is recorded as last object of type # 1 in the collection state of blob # 3 .", + "p0082": "At step 3 , two more core objects 50 are added and are written in Blob # 5 . Core object # 5 links to object # 4 , and is tracked as the last object of Otype # 1 . Object # 6 is, at this stage, unrelated to other core objects 50 and therefore is in an unreferenced state, but is not considered as a purgeable object as it is the last object of the collection.", + "p0083": "At step 4 , three more core objects are added and are written in Blob # 7 . Core objects # 7 , # 8 and # 9 are not tracked as the last object of their Otype. Core objects # 7 , # 8 and # 9 do not contain links and are not linked to. Again, these objects are considered unreferenced, but are not considered as purgeable core objects 50 because they are the last core objects 50 of the collection.", + "p0084": "At step 5 , a core object 50 is added and is written in blob # 10 . Core object # 10 replaces object # 5 . Object # 10 is tracked as the last core object of its Otype , and linked to objects # 4 , # 6 , # 7 and # 8 . This restructures the tree of objects with object # 10 now being the head. Object # 5 , being replaced by object # 10 and object # 9 , now orphaned, are unreferenced and considered obsolete and can be added to the purge candidate list.", + "p0085": "At step 6 , two more core objects 50 are added and are written in Blob # 11 . Object # 12 replaces object # 10 . Object # 12 is tracked as the last object of its Otype , and is linked to objects # 4 , # 6 , and # 11 . This again restructures the tree of objects with core object # 12 now being the head. Core objects # 7 , # 8 and # 10 become obsolete and can be added to the purge candidate list.", + "p0086": "The example of FIG. 6 shows that, as core objects 50 are added, linked, replaced or discarded, the collection state in each blob 60 reflects the precise state of all active core objects 50 in the collection 80 at each of the steps. Such an example also allows easy understanding of how the structure produced using the method for performing data storage 10 can easily be used for time travel , as explained in more details below. In the course of the present description, the term \u201ctime travel\u201d refers to the ability to go back in time and inspect the stored data as it existed in previous states. Using the above described structure, to travel in time and review the state of a collection as it existed at a point in the past, the view of the \u201ctail\u201d of the collection 80 is redefined to be at a given BID 61 a or OID 52 a . Hence, the view of all object trees and core objects 50 also reflects how they existed at the precise point in time where the blob 60 with the given BID 61 a was the last blob 60 . One skilled in the art will therefore understand that time travel can be performed without requiring any precise archiving process or \u201csnapshot\u201d of the stored data to be taken, and that as long as past objects have not yet been purged, there are no restrictions or limits to time travel within the stored data. One skilled in the art will also recognize that redefining the tail BID or OID view of the collection does not change the collection itself, and multiple users may hold different views of the collection at the same time.", + "p0087": "In view of the above, the combination of time travel and optimized blob purge parameters can allow advanced archiving strategies using substantially reduced storage resources. For example, and without being limitative, in an embodiment a continuity of backends 70 can also be used to emulate the behavior of prior art archival \u201csnapshot\u201d functions, while allowing the collection 80 to be time-traveled continuously at any single point in time prior to the data being frozen on an archival backend. In such an embodiment, a primary backend and a secondary backend of a collection can be layered such that all the original blobs are frozen in the primary backend and only the newer or partially purged blobs are recorded in the secondary backend. Hence, while both the primary backend and secondary backend can be used as possible backends 70 for the collection , updates and deletions are only recorded on the secondary backend. A similar strategy can later be used for the primary and secondary backends with regards to a tertiary backend . At this point, the primary, secondary and tertiary backends are used as possible backends for the collection, but only the tertiary backend is subject to object/blob addition and/or deletion.", + "p0088": "One skilled in the art will also understand that full and incremental backup strategies can also be easily implemented using the above described blobs 60 and backends 70 for example, using mirrored layered backends 70 writing blobs 60 to two or more locations simultaneously. Alternatively, blobs 60 can be copied externally from a live backend 70 to a stand-by backend 70 . It will be understood that, given that core objects 50 are immutable and blobs 60 are log-structured, there is no need to synchronize blobs 60 or proceed to a validation of which blobs 60 were modified since the last backup, the only blobs 60 that need to be backed-up or copied being the ones that are new.", + "p0089": "In view of the above, it will be understood that the above-described structure provides a robust arrangement, where the loss or damage to a blob 60 can generally be recovered without causing data loss beyond the data of the specific lost or damaged blob 60 . Indeed, given that the collection management information of each blob 60 includes the state of the collection and its core objects 50 when the blob 60 was written, a preceding blob 60 can generally be used to continue processing a request, even if the damaged or lost blob contained crucial structural information.", + "p0090": "Such a recovery of the information can easily be performed in cases where previous blobs 60 still contain all of the necessary collection management information. However, in cases where garbage collection has been performed and has started cleaning up information, the previous blobs 60 may not readily contain all the information necessary to resume exploring the collection 80 . Hence, in an embodiment, garbage collecting can be configured to retain critical objects, such as directory indices, even if they are technically obsolete, in order to avoid deleting information that may be subsequently relevant in the case of recovery of information following the damage or loss of a blob 60 . Moreover, if garbage collection has been performed and information necessary to resume exploring the collection has been deleted, critical core objects 50 containing links to other core objects 50 can be reconstructed from the referred-to objects.", + "p0091": "In an embodiment, the method for performing data storage 10 can further include a step of forking the collection into two or more distinct sub-collections 25 , wherein each sub-collection 80 can evolve individually while still sharing the same physical blobs and backends for past objects and blobs that they have in common. When a collection 80 is forked, it is substantially the equivalent of taking a prior art \u201csnapshot\u201d or copying the entire collection 80 and then modifying the copy, to the exception that there is no real snapshot or copy but rather a simple shared portion between the forked collections 80 . In an embodiment, in order to proceed with the step of forking the collection 25 , the method for performing distributed data storage 10 includes the step of introducing multiple \u201clast blobs\u201d .", + "p0092": "From the multiple last blobs 60 , each fork defines its own tree of objects. Hence, given the above described structure where the \u201clast blob\u201d is the key holding the state of a collection and all the core objects 50 of the collection together, each fork represents a unique view of the collection 80 and can create/update core objects 50 in this specific sub-collection, without affecting objects or the state of other forks or the base collection 80 . It will be understood that, the method 10 can further include the subsequent step of joining and/or merging the forks, such that the updates performed individually in each sub-collection generated by the corresponding fork can be recombined into a single collection 80 containing a unique chain of core objects 50 and blobs 60 .", + "p0093": "In an embodiment, time travel can be performed in parallel on each individual fork.", + "p0094": "It will be understood that, depending on the embodiment, the forks can be either logical or physical. As will be described in more details below, logical forks are created within the confines of the same backend and share the same blobs, OID and BID namespaces as other forks. Physical forks are implemented in separate backends and each maintain their own unique blobs, OID and BID namespace. One skilled in the art will understand that either fork type can be used to implement the step of forking a collection into multiple sub-collections of the present method 10 .", + "p0095": "One skilled in the art will understand that, since physical forks use distinct backends and namespaces, it is possible for the created physical forks to assign the same OIDs 52 a and BIDs 61 a to the generated objects and blobs. Therefore, when merging physical forks, it will be necessary to recreate the merged objects on the target fork with new unique OID's 52 a . In other words, it is not possible to simply link objects between the generated forks, as is possible with logical forks. In view of the above, two or more physical forks can have colliding BIDs 61 a OIDs 52 a , given that physical forks are fully isolated recorded in the blobs of each fork) and evolve independently from one another. Hence, in operation, the two or more physical forks operate similarly as different collections having only a shared portion, . In order to merge two physical forks, merge helpers are required to create new objects on the target fork corresponding to the objects created on the other physical fork and therefore reintegrate changes applied on another physical fork .", + "p0096": "In an embodiment, the physical forks can be further protected by chain checksums. Such checksum values can be used to validate that physical forks are based on the same common blobs, before engaging a join or merge of the physical forks.", + "p0097": "Referring to FIG. 7 a , an example of forking of a collection in accordance with an embodiment where a logical fork is opened, updated, and subsequently joined back to the base collection is described. Once again, in the example of FIG. 7 a , the evolution of the collection 80 and sub-collections is followed blob 60 by blob 60 . In the example of FIG. 7 a , there are two object types 52 b for the core objects of the collection: Otype # 1 for regular core objects, and Otype #\u2212 5 representing logical fork core objects.", + "p0098": "At step 1 , two core objects 50 are created and are written in blob # 1 . Object # 2 links to object # 1 and is flagged as the last object of Otype # 1 . The object tree with objects # 1 and # 2 is reflected in the state of the collection stored in blob # 1 , with object # 2 as last object of Otype # 1 .", + "p0099": "At step 2 , a logical fork core object is created and an object is created and written in blob # 3 . Object # 4 is added to the collection in the sub-collection of fork # 3 , and is linked to object # 2 . The collection state stored in blob # 3 reflects that blob # 3 belongs to the sub-collection of fork # 3 , with a tree of objects starting with object # 4 . The collection state stored in blob # 3 also reflects that the last fork object is fork # 3 . At this point, the collection 80 includes two sub-collections , both sharing objects # 1 and # 2 , but with the sub-collection of fork # 3 having a larger object tree .", + "p0100": "At step 3 , two new core objects 50 are created in the sub-collection of fork # 3 , and are stored in blob # 5 . Object # 6 is linked to objects # 4 and # 5 , and is flagged as the last object of its Otype . This creates a tree of objects with object # 6 as the head still sharing objects # 1 and # 2 with the sub-collection corresponding to the base collection. At step 3 , fork # 3 is closed by the writing of Fork Close Object # 7 , referring to fork # 3 . Blob # 5 is written under fork # 3 , reflecting the latest state of the fork and its objects.", + "p0101": "At this point, the collection has two forks: fork # 0 , which is the base collection, has a state showing the last object of Otype # 1 is object # 2 , which forms a tree of two objects, and fork # 3 , which is a sub-collection of a closed logical fork, has a tree of objects with object # 6 as its head . The collection state in the sub-collection of fork # 3 also shows that the last fork object has an OID # 7 , thereby showing that fork # 3 has been closed.", + "p0102": "At step 4 , the sub-collection of logical fork # 3 is joined back into the base collection by creating a Fork Base Join core object 50 and written in blob # 20 . The join is performed by updating the collection state to reflect the state of the sub-collection of fork # 3 , thereby automatically integrating all objects from the object tree of the sub-collection of fork # 3 into the main collection . Because the join is done on the base collection, blob # 20 also belongs to the base collection and has a FID # 0 . At this point, the fork history can be examined through fork Object # 20 , but otherwise the collection is considered as having no forks and works like any other standard collection.", + "p0103": "Referring to FIG. 7 b , an example of forking of a collection in accordance with an embodiment where two logical forks are opened, updated, and subsequently merged back to the base collection is described. Once again, in the example of FIG. 7 b , the evolution of the collection 80 and sub-collections is followed blob 60 by blob 60 . In the example of FIG. 7 b , there are once again two object types for the core objects of the collection: Otype # 1 for regular core objects, and Otype #\u2212 5 representing logical fork core objects.", + "p0104": "At step 1 , two core objects 50 are created and are written in blob # 1 . Object # 2 links to object # 1 and is flagged as the last object of Otype # 1 . The object tree with objects # 1 and # 2 is reflected in the state of the collection stored in blob # 1 , with object # 2 as last object of Otype # 1 .", + "p0105": "At step 2 , a logical fork core object is created and a core object is created and written in blob # 3 . Object # 4 is added to the collection in the sub-collection of fork # 3 , and is linked to object # 2 . The collection state stored in blob # 3 reflects that blob # 3 belongs to the sub-collection of fork # 3 , with a tree of objects starting with object # 4 . The collection state stored in blob # 3 also reflects that the last fork object is fork # 3 . At this point, the collection 80 includes two sub-collections , both sharing objects # 1 and # 2 , but with the sub-collection of fork # 3 having a larger object tree .", + "p0106": "At step 3 , a second logical fork core object is created and reserving object OIDs up to # 39 for its own use) and a core object is created and written in blob # 20 . Object # 21 is added to the collection in the sub-collection of fork # 20 , and is linked to object # 2 . The collection state stored in blob # 20 reflects that blob # 20 belongs to the sub-collection of fork # 20 , with a tree of objects starting with object # 21 . The collection state stored in blob # 20 also reflects that the last fork object is fork # 20 . The fork object # 20 indicates that fork # 3 is also active at this time. At this point, the collection 80 includes three sub-collections , all sharing objects # 1 and # 2 , but otherwise having different object heads and trees.", + "p0107": "At step 4 , fork # 3 evolves parallel to fork # 20 . Two core objects 50 are created . Object # 6 is linked to objects # 4 and # 5 , and is flagged as the last object of its Otype . This creates a tree of objects in fork # 3 with object # 6 as the head . At step 4 , fork # 3 is closed by the writing of a Fork Close Object , referring to fork # 3 . Blob # 5 is written under fork # 3 , reflecting the latest state of the fork and its objects.", + "p0108": "At step 5 , fork # 20 evolves parallel to fork # 3 . Three core objects 50 are created and written in blob # 22 . Object # 24 is linked to objects # 21 and # 23 . Object # 24 is flagged as the last object of its Otype . Object # 23 is linked to object # 22 . This creates a tree of objects in fork # 20 with object # 24 as the head . At step 5 , fork # 20 is closed by the writing of a Fork Close Object , referring to fork # 20 and also written in blob # 22 . Blob # 22 is written under fork # 20 , reflecting the latest state of the fork and its objects.", + "p0109": "At step 6 , the sub-collection of logical fork # 3 is joined back into the base collection by creating a Master Fork core object ) and written in blob # 40 . The join is performed by updating the collection state to reflect the state of the sub-collection of fork # 3 , thereby automatically integrating all objects from the object tree of the sub-collection of fork # 3 into the main collection. The Master Fork object # 40 identifies the fork that was joined , other forks that are still active at this time , and the master fork .", + "p0110": "Since the master fork and the fork object # 20 reflect different evolutions of the collection 80 , the content of these two forks cannot simply be joined . Therefore, to integrate the evolution of both forks into a single fork, the two forks must be merged. Hence, at step 7 , a new logical fork is created , with the intent to merge forks # 20 and # 40 . A core object 50 is created . Object # 42 replaces object # 5 of the master fork # 40 and deletes object # 21 of fork # 20 . An additional core object 50 is created and links to objects belonging to three object trees: 1) to newly created object # 42 of the merge fork # 41 ; 2) to object # 4 of the master fork # 40 ; and 3) to object # 23 of fork # 20 . Object 43 is now the new head object of Otype # 1 of the merge fork # 41 . At step 7 , fork # 41 is finally closed by the writing of a Fork Close core object , referring to fork # 41 , and written in blob # 41 . Blob # 41 is written under fork # 40 , and the collection state reflects a tree of objects that merges all elements from forks # 20 and # 40 .", + "p0111": "Given that the merge fork # 40 now represents the state of the collection with all other forks merged into one fork , the sub-collection of logical fork # 41 is joined back into the base collection with a Fork Base Join core object ). The join is performed by updating the collection state to reflect the state of the sub-collection of fork # 41 , thereby automatically integrating all objects from the object tree of the sub-collection of fork # 41 into the main collection . Because the join is done on the base collection, blob # 51 also belongs to the base collection and has a FID # 0 . At this point, the collection is considered as having no forks and works like any other standard collection.", + "p0112": "Referring to FIG. 7 c , an example of forking of a collection in accordance with an embodiment where a physical fork is opened, updated, and subsequently joined back to the base collection is described. Once again, in the example of FIG. 7 c , the evolution of the collection 80 and sub-collections is followed blob 60 by blob 60 . In the example of FIG. 7 c , there are two object types for the core objects of the collection: Otype # 1 for regular core objects, and Otype #\u2212 6 representing Physical Fork core objects.", + "p0113": "At step 1 , two core objects 50 are created and are written in blob # 1 . The object tree with objects # 1 and # 2 is reflected in the state of the collection stored in blob # 1 , with object # 2 as last object of Otype # 1 . In addition, a Physical Fork core object is created . The last object for each Otype are recorded in the collection state in blob # 3 . Given that blob # 1 belongs to the base collection at this point, it is written with a FID # 0 .", + "p0114": "A new backend is assigned to save all the blobs 60 of the new physical fork . It will be understood that any backend type and/or configuration can be used for saving the blobs associated to the new physical fork. In an embodiment, the backend can even be a temporary memory-based fork.", + "p0115": "At step 2 , a core object is created and written in blob # 4 . Object # 4 is added to the collection with a FID # 3 and the physical fork now represents a chain of objects, with OID # 4 as the head, stored in the physical fork, and chained to OID's # 2 and # 1 stored in the base collection. The collection state stored in blob # 3 reflects that blob # 3 belongs to the sub-collection of fork # 3 , with a tree of objects starting with object # 4 . At this point, the collection 80 includes two sub-collections , both sharing objects # 1 and # 2 , but with the sub-collection of fork # 3 having a larger object tree . It will be understood that the physical fork does not affect the base collection, which still includes an object tree with OID # 2 as the head.", + "p0116": "At step 3 , two new core objects 50 are created in the sub-collection of the fork # 3 , and are stored in blob # 5 . Object # 6 is linked to objects # 4 and # 5 , and is flagged as the last object of its Otype . This creates a tree of objects with object # 6 as the head still sharing objects # 1 and # 2 with the sub-collection corresponding to the base collection. At step 3 , fork # 3 is closed by the creation of a Fork Close Object , referring to fork # 3 and written to blob # 5 . Blob # 5 is written under fork # 3 , reflecting the latest state of the fork and its objects. Now that fork # 3 is marked as closed, no other core objects can be written in this physical fork, thereby ensuring that the fork won't be updated afterwards.", + "p0117": "In the example of FIG. 7 c , given that the base collection has not been updated since the physical fork was created, the changes made in the physical fork can be reintegrated back into the main collection with a simple join. In the embodiment shown, this is accomplished in steps 4 and 5 by copying the blobs associated to the physical fork back into the main collection backend. In an alternative embodiment , reintegration of the changes made in the physical fork into the main collection could also be achieved by reconfiguring the backend of the physical fork as being part of the main collection. Such a reconfiguration would achieve the same thing as recopying the blobs associated to the physical fork back into the main collection backend, without blob movement.", + "p0118": "Finally, at step 6 , a Fork Join Core Object is created and written in blob # 8 , in the main collection. The creation of the Fork Join Core Object in the base collection ensures that future blobs will be written with a FID # 0 .", + "p0119": "Referring to FIG. 7 d , an example of forking of a collection in accordance with an embodiment where a physical fork is opened, updated simultaneously with the base collection, and subsequently merged back with the base collection. Once again, in the example of FIG. 7 d , the evolution of the collection 80 and sub-collections is followed blob 60 by blob 60 . In the example of FIG. 7 d , there are two object types for the core objects of the collection: Otype # 1 for regular core objects, and Otype #\u2212 6 representing Physical Fork core objects.", + "p0120": "At step 1 , two core objects 50 are created and are written in blob # 1 . Object # 2 links to object # 1 and is flagged as the last object of Otype # 1 . The object tree with objects # 1 and # 2 is reflected in the state of the collection stored in blob # 1 , with object # 2 as last object of Otype # 1 . In addition, a physical Fork core object is created . The last object for each Otype are recorded in the collection state in blob # 3 . Given that blob # 1 belongs to the base collection at this point, it is written with a FID # 0 .", + "p0121": "Once again, a new backend is assigned to save all the blobs 60 of the new physical fork . As stated above, it will be understood that any backend type and/or configuration can be used for saving the blobs associated to the new physical fork.", + "p0122": "At step 2 , a core object is created and written in blob # 4 . Object # 4 is added to the collection with a FID # 3 and the physical fork now represents a chain of objects, with OID # 4 as the head, stored in the physical fork, and chained to OID # 1 stored in the base collection. The collection state stored in blob # 3 reflects that blob # 3 belongs to the sub-collection of fork # 3 , with a tree of objects starting with object # 4 . At this point, the collection 80 includes two sub-collections . The sub-collection of fork # 3 has an object tree including object # 4 linked to object # 1 ). It will be understood that the physical fork does not affect the base collection, which still includes an object tree with OID # 2 as the head.", + "p0123": "At step 3 , two core objects are created in the base collection and written in blob # 4 of the base collection. It will be understood that, the newly created core objects and blobs are different than those in the physical fork , given that the physical fork and the base collection now have completely diverged . The base collection now has a tree of core objects with OID # 5 , and OID # 4 being different than the core object having object OID # 4 in the physical fork . The two forks , have different FIDs and use different backends and therefore are isolated from one another, while still sharing common core object # 1 that existed before the collection was forked.", + "p0124": "At step 4 , two core objects are created in the physical fork and written in blob # 5 of the physical fork . Once again it must be understood that object # 5 of physical fork # 3 is different than object # 5 of the base collection. Object # 6 of fork # 3 is linked to objects # 4 and # 5 , and is flagged as the last object of its Otype . At step 4 , fork # 3 is closed by the writing of a Fork Close Object , referring to fork # 3 , and written in blob # 5 . Blob # 5 is written under fork # 3 , reflecting the latest state of the fork and its objects. The collection state in blob # 5 reflects object OID # 6 and the fork close object # 7 as last objects of their Otypes. Now that fork # 3 is marked as closed, no other core objects may be written in this physical fork, thereby ensuring that the fork won't be updated afterwards.", + "p0125": "Since the base collection and the physical fork were updated at the same time, the forks cannot simply be joined; to reintegrate changes made in the physical fork back into the base collection, the forks need to be merged. At step 5 , the base collection and the physical fork are merged using a helper . The helper reconstructs the payload of the core objects of the physical fork in the base collection in newly created core objects. In the embodiment shown, core object # 4 of fork # 3 is recreated under core object # 6 of the base collection, core object # 5 of fork # 3 is recreated under core object # 7 of the base collection, and core object # 6 of fork # 3 is recreated under core object # 8 of the base collection, with each newly created core object being linked to corresponding existing objects of the base collection or being replaced/deleted, as necessary. After the merge, object # 1 has two parents . This is a valid construct and there are no theoretical limits to the number of links an object can be the child of. A physical Fork Merge is written in the base collection to record the fact that the physical fork # 3 was merged into the base collection and is no longer used. The blobs including all merged core objects are written under FID # 0 , as those blobs now belong to the base collection.", + "p0126": "In an embodiment , where the merge is complex and involves many blobs, a logical fork can be initially created in the main collection and used to record the merging process before being joined back to the main collection. The use of a logical fork helps to prevent the merge operation from affecting the main collection's view of objects until all updates are integrated.", + "p0127": "In view of the above, the combination of immutability and unique OIDs 52 a of the core objects 50 with BIDs 61 a being derivative of the OIDs 52 a of the core objects 50 it contains, allow the blobs 60 to be highly stable, autonomous and freely mobile data containers that need no central control, tracking, indices or maps.", + "p0128": "Because core objects 50 are themselves immutable and OIDs 52 a are permanently unique, blobs 60 are by extension essentially immutable. Hence, given that the core objects 50 in a blob 60 cannot change, either by content or by OID 52 a , there is no need to define or store which \u201cversion\u201d of a blob 60 copied or replicated to another location contains the latest version of a given core object 50 ; all blobs 60 with a given BID 61 a are equivalent.", + "p0129": "Because BIDs 61 a are derived from the OIDs 52 a of the core objects 50 of the blob 60 , it can be determined which blob 60 contains a given core object 50 using only the OID 52 a of the sought core object 50 , thereby removing the burden of having to maintain indices or maps as to which blobs 60 contain which core objects 50 .", + "p0130": "Because the BID 61 a conveys not only the identity of the blob 60 , but also, by extension, the identity, state and contents of all objects 50 within the blob 60 , this prevents the need to share information regarding the content of a blob 60 with the backends 70 and allows the entire blob to be encrypted and totally hidden, including any control information or other metadata. Because blobs 60 are totally sealed, encrypted containers of data, they can be securely copied by any mechanism or agent without them needing to know anything about the contents or nature of the blob 60 . Hence, backends 70 can move and copy blobs freely from one backend to another without understanding or confirming the contents of a blob 60 , even if infrastructures of the backends 70 are different. The only things that need to be exchanged are BIDs 61 a and the content of blobs 60 .", + "p0131": "Finally, because BIDs 61 a are derived from the OIDs 52 a of the sought core objects 50 and can be located using a short list of BIDs 61 a held by storage backends 70 , the exact location of the core objects 50 is automatically disclosed in real-time through BID derivation. No tracking or indexing of the blobs 60 is required. This allows free movement or scattering of the blobs 60 between backends 70 , without notification to a central unit. The storage backends 70 are the only components required to know which blobs 60 they contain. As long as a blob 60 with a given BID 61 a is available somewhere, the data is readily available. Moreover, blobs 60 can be freely replicated to any storage medium or service, without central coordination, to increase the availability and integrity of the data. Any copy of a blob, wherever it is located, can be used.", + "p0132": "In view of the above, the described structure also allows collections 80 to be shared between users in peer-to-peer mode, without requiring the use of a central coordination unit such as a central server or the like, even for updates. Collection sharing is possible in view of the structure of the blobs 60 and log-structured core objects 50 , by application of collision Integrity principles. For example, in an embodiment where peer to peer sharing of a collection is performed between two or more users, if the two or more users attempt to add objects and write to the same blob 60 at the same time, a collision is detected and only one of the user will succeed in writing the blob 60 , the others failing as a result of the collision. Such failure of the writing of the blob 60 implies that the collection 80 has been updated. Hence, each of the users for which the writing of the blob failed needs to refresh its view of the collection with the new state of the collection and reconstruct the objects under a new blob 60 . Subsequently, it can then resubmit the update in the new blob . Of course, if a collision is detected again, the process is repeated until it succeeds for the precise user. In alternative embodiments, additional mechanisms can be used, in connection with collision Integrity principles, or separately, in order to limit or alleviate the occurrence of collisions. For example and without being limitative, logical forks can be used for each user to reduce the risks and frequency of collisions. One skilled in the art will recognize that, for collision detection to work, possibly colliding blobs must be stored in the same backend shared by all users. However, blobs associated to physical forks or reserved OID ranges in logical forks, which cannot collide, may be stored in any backend not necessarily used or shared by all users. One skilled in the art will also recognize that, in an embodiment, a lock server could be used instead of collision detection, in order to ensure sharing integrity. In another alternative embodiment, peer-to-peer sharing can also be done without using any shared common backends 70 nor relying on collision detection. This is known as detached backends sharing where the collection is simply configured so that each user has its own physical backends 70 with a private copy of the blobs 60 and can therefore read and update their own copy of the collection 80 separately. When clients reconnect, their detached collections can be joined/merged as if they were physical forks in order to reintegrate changes that were performed while they were detached.", + "p0133": "The method for performing data storage 10 having been described above, a system 100 for performing the above described distributed data storage will now be described.", + "p0134": "With reference to FIGS. 8 and 9 , in an embodiment, the system 100 includes a core engine module 110 in data communication with data sources such as engine modules 104 and/or applications 106 , requesting data storage. In general terms, the core engine module 110 is the engine providing the basic services under which all other services rely upon in order to perform the distributed data storage. In other words, the core engine module 110 is the main module of the system 100 which manages the creation, update, deletion of the objects, blobs, collections and backends from the collection, performs blob compression/decompression , performs blob encryption/decryption , performs collection management, and the like.", + "p0135": "The core engine module 110 , engines modules 104 , and/or applications 106 and/or sub-components thereof can be part of a local computing system . One skilled in the art will however understand that, in an embodiment, the core engine module 110 , engines 104 and/or applications 106 and/or sub-components thereof can be distributed on separate computing unit in data communications with one another, for example over a network.", + "p0136": "In an embodiment, the core engine module 110 includes a core Application Programming Interface module 112 through which additional engine modules 104 and applications 106 can request storage services. Applications 106 are external programs which use the system 100 to store data. In an embodiment, applications 106 use the core engine API module 112 to structure their data as discrete objects. In alternative embodiments , the applications 106 can also use the database engine module 134 , which will be described in more details below, to store its data as table records, or use the filesystem engine module 132 , which will also be described in more details below, directly or indirectly, to work with files. In an embodiment, the core API module 112 is operative to open collections, read, write and delete core objects and the like.", + "p0137": "In an embodiment, the core engine module 110 also includes a collection manager module 114 in communication with the core API module 112 . The collection manager module 114 handles all high-level requests and manages the core objects. For example, the collection manager module 114 assigns compliant OIDs to created objects and, in an embodiment, assigns appropriate object identification information to the objects, as discussed above. The collection manager module 114 also includes the garbage collector operative to identify and purge obsolete objects, i.e. objects which are unreferenced and are purgeable according to the defined purge parameters, as discussed above in relation to the corresponding method.", + "p0138": "The collection manager module 114 communicates with a blob manager module 116 for generating and managing blobs. The blob manager module 116 generates blobs and assigns BID, locates blobs, coordinates access to multiple backends, and implements asynchronous inputs/outputs with the backends.", + "p0139": "The collection manager module 114 and blob manager module 116 implement the above described central features of the objects and blob, namely the object immutability, permanently unique object IDs , derived blob IDs and predictable BIDs, which allow the capacity to subsequently freely store, move, copy and scatter data from one backend 170 to another without using indices, central control, tracking or synchronization and to share that data in peer-to-peer mode with updates without using locks or a central coordination server.", + "p0140": "The blob manager module 116 in turn communicates with an independent compressor module 118 to compress and decompress the contents of the blobs. In an embodiment, the compressor module 118 can implement multiple compression algorithms and encapsulates each blob with a compression tag to identify the compression algorithm used for subsequent decompression when a blob is read back for decompression thereof. One skilled in the art will understand that, in an embodiment where no compression is applied to the blobs, the core engine module 110 can be free of compressor module 118 .", + "p0141": "The core engine module 110 also includes an encryptor module 120 communicating with the compressor module 118 . Similarly to the compressor module 118 , the encryptor module 120 can implement a plurality of encryption algorithms and can therefore apply different encryptions to different blobs, thereby enhancing security of the overall system 100 . The encryptor module 120 can also encapsulate the blobs with an encryption tag indicating how each specific blob can be decrypted when read back from the backends. Once again, one skilled in the art will understand that, in an embodiment where no encryption is applied to the blobs, the core engine module 110 can be free of encryptor module 120 .", + "p0142": "In the embodiment shown, the encryptor module 120 is in data communication with the backends 170 . As mentioned above, multiple backends can be associated to each collection and the blobs can be autonomously distributed and/or scattered across multiple backends for storage thereof, in order to perform distributed data storage of the stored data. In an embodiment where the core engine module 110 is free of encryptor module 120 and/or compressor module 118 , the backends 170 can be in data communication directly with the blob manager module 116 .", + "p0143": "In the embodiment shown, the compressor module 118 , encryptor module 120 and backends 170 work as stream handlers, freely passing data between themselves to read and write blobs. As will be easily understood, at the end of the stream, backends 170 are where the blobs are actually physically stored, in the cloud and/or on local physical storage devices.", + "p0144": "Still referring to FIGS. 8 and 9 , in an embodiment, the system 100 includes additional engine modules such as the index engine module 130 , filesystem engine module 132 and database engine module 134 , which will be described in more details below. The index engine module 130 , filesystem engine module 132 and database engine module 134 are operative to provide file and database services to users, operating systems and applications.", + "p0145": "The index engine module 130 provides indexing services to other engine modules such as the filesystem engine module 132 and database engine module 134 and to applications 106 , such that data stored in a collection can be easily organized and located. In other words, the index engine module 130 stores key/value pairs in order to allow values to be efficiently stored and retrieved by key. For example and without being limitative, in an embodiment, for the index engine module 130 , indices can be saved as regular core engine objects, linked together by OIDs, with a head object holding the tree of all objects for a given index. In an embodiment, indices are stored internally as log-structured B+Trees, but one skilled in the art will understand that, in alternative embodiments, other data structure allowing high speed searches, sequential access, insertions, and deletions could also be used to store indices. For example and without being limitative, in an embodiment, the index engine module 130 is used, amongst others, by the core engine module 110 to manage object aliases and Virtual IDs for the objects thereof. The index engine module 130 can also be used to maintain private indices which can contain any form of key/value pairs. For example and without being limitative, in an embodiment, the private indices can be used by engine modules for efficiently managing key/value pairs, for example to hold a dynamic dictionary, a list of named items, or the like.", + "p0146": "The filesystem engine module 132 implements file storage using the above described structure of objects and blobs. In an embodiment, the filesystem engine module 132 is in data communication with the core engine module 110 and includes APIs through which well-known filesystem primitives such as readdir, open, read/write and close can be used by applications, user computing device or the like, to store and retrieve files, directories, metadata and associated structures. For example and without being limitative, in an embodiment, the filesystem engine module 132 can be connected directly to an operating system interface , to allow the storage of files by users as if they were local files. The filesystem engine module 132 , therefore allows users to store and securely scatter their files on multiple backends, such as cloud storage providers or other mediums without central coordination, using the core engine module 110 . In view of the above, one skilled in the art will understand that, in an embodiment where the backend 170 of the core engine module 110 is a physical disk or other local storage media, the system 100 can be used directly as a true, end-to-end, filesystem for the user's computing device, with all the features and advantages of the distributed storage facilities provided by the system 100 .", + "p0147": "The database engine module 134 provides database services to applications through published interfaces , and stores the corresponding structures as core objects and index engine private indices. Therefore, the database engine module 134 collaborates with the index engine module 130 and the core engine module 110 in order to perform data storage for databases. The use of the index engine module 130 and the core engine module 110 of the present system 100 allows databases to be fully distributed and scattered on different backends 170 such as, multiple cloud storage providers, shared storage systems and local media, with all the benefits of the above-described system 100 .", + "p0148": "The above described method and system for distributed storing of data allows to freely store, move, copy and scatter data from one storage location to another without using indices, central control, tracking or synchronization, and to share that data in peer-to-peer mode with updates without using locks or a central coordination server. Such characteristics allows data to be simultaneously stored on local disks and cloud storage, updated in real-time, moved or copied from one medium to another without notification to a central system, and have the core engine automatically locate and retrieve the required information at any time with high integrity.", + "p0149": "Several alternative embodiments and examples have been described and illustrated herein. The embodiments of the invention described above are intended to be exemplary only. A person skilled in the art would appreciate the features of the individual embodiments, and the possible combinations and variations of the components. A person skilled in the art would further appreciate that any of the embodiments could be provided in any combination with the other embodiments disclosed herein. It is understood that the invention may be embodied in other specific forms without departing from the central characteristics thereof. The present examples and embodiments, therefore, are to be considered in all respects as illustrative and not restrictive, and the invention is not to be limited to the details given herein. Accordingly, while specific embodiments have been illustrated and described, numerous modifications come to mind without significantly departing from the scope of the invention as defined in the appended claims.", + "features": { + "F1": "A computer-implemented method for performing data storage includes receiving data to be stored from a data source.", + "F2": "The method involves segmenting the data into immutable core objects, each being written into a collection and assigned a unique object ID.", + "F3": "The method involves grouping the immutable objects into blobs, each including at least one of the objects, with each blob assigned a unique blob ID derived from the object ID of at least one of the objects included.", + "F4": "The last blob of the collection is identifiable as such.", + "F5": "The method involves defining a state of the collection by linking at least one subset of the core objects of the collection to one another, with the last object of each subset being the object head for the corresponding subset, and storing identification information of the head object.", + "F6": "The method involves storing each blob onto at least one backend.", + "F7": "Storing identification information of the head object involves storing it in the last blob of the collection.", + "F8": "The method involves selectively retrieving corresponding blobs from one of the at least one backend upon request of the stored data.", + "F9": "The method involves encrypting the blobs prior to storing them onto the at least one backend and decrypting the blobs after retrieving them from one of the at least one backend.", + "F10": "The method involves compressing the blobs prior to encrypting them and decompressing the blobs after decrypting them.", + "F11": "Each core object has an object type, and maintaining the state of the collection involves linking active objects of each object type, with the last object being the head object for the corresponding object type.", + "F12": "Storing each blob onto at least one backend includes storing the blobs onto distributed backends.", + "F13": "The method comprises a step of forking the collection by introducing multiple last blobs identifiable as the last blob of each forked collection.", + "F14": "The method includes merging the forked collection into one of a collection and another fork of a collection.", + "F15": "A computer readable memory records statements and instructions for execution by a computer to carry out the method of claim 1.", + "F16": "A data storage system comprises a core engine module configured to receive data to be stored, store the data onto at least one backend, and retrieve the data from the backend.", + "F17": "The core engine module generates and manages at least one collection comprising core objects, each including data to be stored and being immutable.", + "F18": "Each core object in the collection is assigned a unique object ID, with at least one subset linked to one another and the last object being the object head for the corresponding subset.", + "F19": "The collection includes blobs, each comprising at least one object and assigned a unique blob ID derived from the object ID of at least one object included.", + "F20": "The last blob of the collection is identifiable as such and includes identification information of the head object of each subset.", + "F21": "The backend stores the blobs and allows them to be retrieved using the corresponding blob ID.", + "F22": "The core engine module comprises a core API module configured to open collections and read, write, and delete core objects.", + "F23": "A collection manager module generates and manages the core objects.", + "F24": "A blob manager module generates and manages the blobs.", + "F25": "The core engine module includes an encryptor module to encrypt blobs prior to storage and decrypt them after retrieval.", + "F26": "A compressor module compresses the blobs prior to encryption and decompresses them after decryption.", + "F27": "Link data representing the links between core objects of each subset is stored in at least a portion of the core objects.", + "F28": "The unique object ID for each core object is assigned sequentially.", + "F29": "The unique blob ID of each blob corresponds to the object ID of the first core object of the blob.", + "F30": "Each core object includes one of an object alias and a virtual object ID.", + "F31": "An index engine performs indexing services to facilitate the organization and location of data stored in a collection.", + "F32": "The backend of each collection comprises multiple distributed backends for distributed storage of the blobs." + } + } + }, + { + "FAN": "72214279", + "CPC": [ + "G06F-016/00", + "G06F-016/951", + "G06F-017/30/864", + "H04L-012/46/33", + "H04L-045/20", + "H04L-045/306", + "H04L-045/42", + "H04L-045/54", + "H04L-045/74", + "H04L-067/327", + "H04L-067/63", + "H04L-069/22" + ], + "Content": { + "title": "Method and apparatus for processing modified packet", + "pa01": "A method and an apparatus for processing a modified packet. A first routing device obtains a first packet sent by a second routing device, where the first packet carries a first correspondence, creates a forwarding entry that includes the first correspondence, receives a modified packet and determines an ID of a destination service function of the modified packet, queries the forwarding entry according to the ID of the destination SF, and determines that an SF identified by the ID of the destination SF is an SF connected to a second routing device, and forwards the modified packet to the second routing device.", + "c-en-0001": "A method, comprising: obtaining, by a first routing device, a first packet sent by a second routing device, wherein the first packet carries a first correspondence, wherein the first correspondence comprises a correspondence between network address information of the second routing device and an identifier of a service function node; storing, by the first routing device, the first correspondence; receiving, by the first routing device, a second packet, wherein the second packet carries a service chain ID and an ID of a source SF node; determining, by the first routing device, an ID of the SF node to process the second packet according to the service chain ID, further according to the ID of the source SF node and further according to a correspondence associated with the service chain ID, the ID of the source SF and the ID of the SF node; and forwarding, by the first routing device, the second packet to the second routing device according to the first correspondence.", + "c-en-0002": "The method according to claim 1 , wherein the method further comprises: obtaining, by the first routing device, a second correspondence, wherein the second correspondence is a correspondence between network address information of the first routing device and an ID of an SF node connected to the first routing device; and sending, by the first routing device, a third packet to a third routing device connected to the first routing device, wherein the third packet carries the second correspondence.", + "c-en-0003": "The method according to claim 2 , wherein the method further comprises: sending, by the first routing device, a fourth packet to the third routing device connected to the first routing device, wherein the fourth packet carries the first correspondence, and wherein the third routing device is different from the second routing device.", + "c-en-0004": "The method according to claim 1 , wherein the first packet obtained by the first routing device further carries a third correspondence, wherein the third correspondence comprises a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and the network type comprises a user datagram protocol network or a generic routing encapsulation network.", + "c-en-0005": "The method according to claim 4 , wherein the forwarding the second packet comprises: forwarding, by the first routing device, the second packet to the second routing device according to the first correspondence using a forwarding tunnel that corresponds to the network type of the network of the second routing device, wherein the forwarding tunnel is determined according to the third correspondence.", + "c-en-0006": "The method according to claim 1 , wherein the SF node is not connected to the first routing device.", + "c-en-0007": "An apparatus, comprising: a non-transitory computer readable medium; and a processor that is connected to the non-transitory computer readable medium, wherein the non-transitory computer readable medium stores a program to be executed by the processor, the program including instructions to: obtain a first packet sent by a second routing device, wherein the first packet carries a first correspondence, wherein the first correspondence comprises a correspondence between network address information of the second routing device and an identifier of a service function node; store the first correspondence; receive a second packet, wherein the second packet carries a service chain ID and an ID of a source SF node; determine an ID of the SF node to process the second packet according to the service chain ID, further according to the ID of the source SF node and further according to a correspondence associated with the service chain ID, the ID of the source SF and the ID of the SF node; and forward the second packet to the second routing device according to the first correspondence.", + "c-en-0008": "The apparatus according to claim 7 , wherein the program further include instructions to: obtain a second correspondence, wherein the second correspondence is a correspondence between network address information of the apparatus for processing the packet and an ID of an SF node connected to the apparatus for processing the packet; and send a third packet to a third routing device connected to the apparatus for processing the packet, wherein the third packet carries the second correspondence.", + "c-en-0009": "The apparatus according to claim 8 , wherein the program further include instructions to: send a fourth packet to the third routing device connected to the apparatus for processing the packet, wherein the fourth packet carries the first correspondence, and the third routing device is different from the second routing device.", + "c-en-0010": "The apparatus according to claim 7 , wherein the first packet carries a third correspondence, wherein the third correspondence comprises a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and wherein the network type comprises a user datagram protocol network or a generic routing encapsulation network.", + "c-en-0011": "The apparatus according to claim 10 , wherein the program further include instructions to: forward the second packet to the second routing device according to the first correspondence using a forwarding tunnel that corresponds to the network type of the network of the second routing device, wherein the forwarding tunnel that is corresponding to the network type of the network of the second routing device is determined according to the third correspondence.", + "c-en-0012": "The apparatus according to claim 7 , wherein the SF node is not connected to the first routing device.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a continuation of International Application No. PCT/CN2015/086801, filed on Aug. 12, 2015, which claims priority to Chinese Patent Application No. 201410401433.8, filed on Aug. 14, 2014. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties.", + "p0003": "TECHNICAL FIELD", + "p0004": "The present invention relates to the communications field, and in particular, to a method and an apparatus for processing a modified packet.", + "p0005": "BACKGROUND", + "p0006": "Generally, multiple packets forwarded in a network belong to one or more service flow types. For example, service flows of different types need to be processed by using different service function nodes.", + "p0007": "In the prior art, a procedure of processing a service packet is generally: when a network receives a packet sent by a user, a classifier determines, according to a preset classification rule, a service flow type to which the packet belongs, and then selects a proper service chain according to the determined service flow type, to process the packet. FIG. 1 is used as an example. FIG. 1 is a schematic diagram of a network for service packet forwarding. The network shown in FIG. 1 includes a routing device 1 , an SF 1 , an SF 2 , an SF 3 , a routing device 2 , an SF 4 , and an SF 5 , where the SF 1 , the SF 2 , and the SF 3 are connected to the routing device 1 , and the SF 4 and the SF 5 are connected to the routing device 2 . When a packet reaches a classifier, the classifier determines a service flow type to which the packet belongs. It is assumed that a service flow of this type needs to be processed by using the SF 1 and the SF 4 successively, the classifier determines that a path that the service packet passes through is: the classifier\u2192the routing device 1 \u2192the SF 1 \u2192the routing device 1 \u2192the routing device 2 \u2192the SF 4 \u2192the routing device 2 . The classifier adds a service chain header to the packet, to obtain a modified packet. The service chain header includes a determined-service-chain identifier . The classifier forwards the modified packet, so that the modified packet passes through all nodes on the service chain successively, and processing of the modified packet is completed. To forward the modified packet on a service chain, the routing device 1 and the routing device 2 need to know not only a correspondence between an ID of an SF and a service chain ID, but also an ID of a next-hop SF on the service chain and network address information of a routing device connected to the next-hop SF. That is, each routing device needs to create a forwarding entry. The forwarding entry includes a correspondence between the service chain ID, the ID of the SF, and network address information of a routing device connected to the SF. The forwarding entry is centrally configured and delivered by a controller.", + "p0008": "A main function of the controller in the network is monitoring a relationship between each SF and the service chain, but not concerning related information of a routing device. However, to configure a forwarding entry for the routing device, the controller needs to additionally obtain network address information of the routing device. Consequently, the controller needs to additionally consume a network resource when configuring the forwarding entry for the routing device, and extra data processing load is brought to the controller.", + "p0009": "SUMMARY", + "p0010": "To resolve the foregoing technical problem, embodiments of the present invention provide a method and an apparatus for processing a modified packet, and a controller is no longer required to additionally obtain network address information of a routing device.", + "p0011": "According to a first aspect, an embodiment of the present invention provides a method for processing a modified packet, including obtaining, by a first routing device, a first packet sent by a second routing device, where the first packet carries a first correspondence, the first correspondence includes a correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and the second routing device is connected to the first routing device. The method further includes creating, by the first routing device, a forwarding entry, where the forwarding entry includes the first correspondence, and the network address information of the second routing device in the forwarding entry indicates address information of a next-hop routing device of the first routing device on a service chain, receiving, by the first routing device, a modified packet, and determining an ID of a destination SF of the modified packet, querying, by the first routing device, the forwarding entry according to the ID of the destination SF, obtaining the network address information of the second routing device from the first correspondence that matches the ID of the destination SF, and determining that an SF identified by the ID of the destination SF is the SF connected to the second routing device, and forwarding, by the first routing device, the modified packet to the second routing device according to the network address information of the second routing device.", + "p0012": "In a first possible implementation manner of the first aspect, the method further includes obtaining, by the first routing device, a second correspondence, where the second correspondence is a correspondence between network address information of the first routing device and an ID of an SF connected to the first routing device, and sending, by the first routing device, a second packet to a third routing device connected to the first routing device, where the second packet carries the second correspondence.", + "p0013": "With reference to the first aspect or the first possible implementation manner of the first aspect, in a second possible implementation manner, the method further includes sending, by the first routing device, a third packet to the third routing device connected to the first routing device, where the third packet carries the first correspondence, and the third routing device is different from the second routing device.", + "p0014": "With reference to the first aspect or the first or the second possible implementation manner of the first aspect, in a third possible implementation manner, the first packet obtained by the first routing device further carries a third correspondence, where the third correspondence includes a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and the network type includes a UDP network or a GRE network, and the forwarding entry created by the first routing device further includes the third correspondence.", + "p0015": "With reference to the third possible implementation manner of the first aspect, in a fourth possible implementation manner, the forwarding, by the first routing device, the modified packet to the second routing device according to the network address information of the second routing device includes forwarding, by the first routing device, the modified packet to the second routing device according to the network address information of the second routing device by using a forwarding tunnel that is corresponding to the network type of the network of the second routing device, where the forwarding tunnel that is corresponding to the network type of the network of the second routing device is determined according to the third correspondence.", + "p0016": "According to a second aspect, an embodiment of the present invention provides an apparatus for processing a modified packet, including a receiving unit, configured to obtain a first packet sent by a second routing device, where the first packet carries a first correspondence, the first correspondence includes a correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and the second routing device is connected to the apparatus for processing a modified packet, a processing unit, configured to create a forwarding entry, where the forwarding entry includes the first correspondence, and the network address information of the second routing device in the forwarding entry indicates address information of a next-hop routing device, on a service chain, of the apparatus for processing a modified packet, where the receiving unit is further configured to receive a modified packet, and determine an ID of a destination SF of the modified packet, and the processing unit is further configured to: query the forwarding entry according to the ID of the destination SF, obtain the network address information of the second routing device from the first correspondence that matches the ID of the destination SF, and determine that an SF identified by the ID of the destination SF is the SF connected to the second routing device, and a sending unit, configured to forward the modified packet to the second routing device according to the network address information of the second routing device.", + "p0017": "In a first possible implementation manner of the second aspect, the receiving unit is further configured to obtain a second correspondence, where the second correspondence is a correspondence between network address information of the apparatus for processing a modified packet and an ID of an SF connected to the apparatus for processing a modified packet, and the sending unit is further configured to send a second packet to a third routing device connected to the apparatus for processing a modified packet, where the second packet carries the second correspondence.", + "p0018": "With reference to the second aspect or the first possible implementation manner of the second aspect, in a second possible implementation manner, the sending unit is further configured to send a third packet to the third routing device connected to the apparatus for processing a modified packet, where the third packet carries the first correspondence, and the third routing device is different from the second routing device.", + "p0019": "With reference to the second aspect or the first or the second possible implementation manner of the second aspect, in a third possible implementation manner, the first packet obtained by the receiving unit further carries a third correspondence, where the third correspondence includes a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and the network type includes a UDP network or a GRE network, and the forwarding entry created by the processing unit further includes the third correspondence.", + "p0020": "With reference to the third possible implementation manner of the second aspect, in a fourth possible implementation manner, the sending unit is further configured to forward the modified packet to the second routing device according to the network address information of the second routing device by using a forwarding tunnel that is corresponding to the network type of the network of the second routing device, where the forwarding tunnel that is corresponding to the network type of the network of the second routing device is determined according to the third correspondence.", + "p0021": "It may be learned from the foregoing technical solutions that a forwarding entry used for forwarding a modified packet between routing devices does not need to be additionally configured for and delivered to a first routing device by a controller. The first routing device may obtain a first packet sent by a second routing device connected to the first routing device, obtain a first correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and create the forwarding entry that includes the first correspondence, where the first correspondence is carried in the first packet. Therefore, when the first routing device needs to forward, to the second routing device, the modified packet whose ID of a destination SF is the ID of the SF connected to the second routing device, the first routing device may query the forwarding entry, and obtain the network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0022": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0023": "To describe the technical solutions in the embodiments of the present invention more clearly, the following briefly describes the accompanying drawings required for describing the embodiments. Apparently, the accompanying drawings in the following description show merely some embodiments of the present invention, and persons of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.", + "p0024": "FIG. 1 is a schematic diagram of a network for forwarding a modified packet;", + "p0025": "FIG. 2 is a flowchart of a method for processing a modified packet according to an embodiment;", + "p0026": "FIG. 3 is a schematic diagram of a network for forwarding a modified packet according to an embodiment;", + "p0027": "FIG. 4 is a flowchart of a method for creating a forwarding entry according to an embodiment;", + "p0028": "FIG. 5 is a flowchart of a method for creating a forwarding entry according to an embodiment;", + "p0029": "FIG. 6 is a structural diagram of an apparatus for processing a modified packet according to an embodiment; and", + "p0030": "FIG. 7 is a schematic diagram of a hardware structure of a first routing device according to an embodiment.", + "p0031": "DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS", + "p0032": "To describe the technical solutions in embodiments of the present invention more clearly, how to forward a modified packet on a service chain is first described. FIG. 1 is used as an example. A service chain of a packet includes an SF 1 and an SF 4 successively, and a service chain ID is 55. A specific implementation manner is: a classifier determines, by analyzing a service flow type included in the packet, that a path that the packet passes through is: the classifier\u2192a routing device 1 \u2192the SF 1 \u2192the routing device 1 \u2192a routing device 2 \u2192the SF 4 \u2192the routing device 2 . The classifier adds a service chain header to the packet to obtain a modified packet. The service chain header may include the service chain ID and an ID of an SF, and the ID of an SF may be an ID of a source SF or an ID of a destination SF. When the ID of an SF is an ID of a destination SF, that is, an ID of the SF 1 on the service chain, the ID of the destination SF refers to an ID of a next-hop SF of the current modified packet. When the modified packet reaches the routing device 1 , the routing device 1 sends, according to the service chain ID and the ID of the SF 1 , the modified packet to the SF 1 connected to the routing device 1 . After the SF 1 processes the modified packet, the modified packet is returned to the routing device 1 . The routing device 1 determines an ID of the next-hop SF 4 by using the service chain ID, determines a corresponding forwarding entry by using the service chain ID and the ID of the SF 4 , and matches network address information of the routing device 2 that is corresponding to the SF 4 . The routing device 1 sends the modified packet to the routing device 2 according to the network address information of the routing device 2 . The forwarding entry includes a correspondence between a service chain ID, an ID of an SF, and network address information of a routing device, and may be shown in Table 1.", + "p0033": "TABLE 1 Service Network address information of a routing chain ID ID of an SF device connected to a next-hop SF 55 ID of an SF 1 Network address information of a routing device 1 55 ID of an SF 4 Network address information of a routing device 2", + "p0034": "A controller may be responsible for configuring the forwarding entry for the routing device and delivering the forwarding entry to the routing device, but the controller needs to additionally consume a network resource to obtain the network address information of the routing device, which brings extra data processing load. In the embodiments of the present invention, the forwarding entry used for forwarding the modified packet between routing devices may not need to be additionally configured for and delivered to a first routing device by the controller. The first routing device may obtain, by using a first packet sent by a second routing device connected to the first routing device, a first correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and create the forwarding entry that includes the first correspondence, where the first correspondence is carried in the first packet. Therefore, when the first routing device needs to forward, to the second routing device, the modified packet whose ID of a destination SF is the ID of the SF connected to the second routing device, the first routing device may query the forwarding entry, and obtain the network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0035": "In addition to creating the forwarding entry for the second routing device according to the first packet sent by the second routing device, the first routing device may further send, by using a second packet, a second correspondence to a third routing device connected to the first routing device, and the second correspondence includes a correspondence between an ID of an SF connected to the first routing device and network address information of the first routing device. The third routing device may create a forwarding entry according to the second correspondence in the second packet, and the forwarding entry may be understood as being for the first routing device. The first routing device may further send the first correspondence obtained from the second routing device to the third routing device by using a third packet, so that the third routing device may further create a forwarding entry, and the forwarding entry may be understood as being for the second routing device. In a manner of diffusing, between routing devices, a correspondence between an ID of an SF and a routing device connected to the SF, the routing device may not only obtain a correspondence between network address information of a connected routing device and an ID of an SF connected to the connected routing device, but also obtain a correspondence between network address information of another connected routing device in a network topology and an ID of an SF connected to the another routing device, and accordingly create a forwarding entry for the another routing device. Therefore, compatibility of the routing device in the network topology and a capability of forwarding a modified packet are improved.", + "p0036": "In addition, the first packet received by the first routing device may further include a third correspondence, and the third correspondence includes a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, so that the forwarding entry created by the first routing device for the second routing device further includes the third correspondence. When forwarding the modified packet to the second routing device, the first routing device may select a tunnel corresponding to the network type of the second routing device to forward the modified packet, where the network type of the second routing device is a user datagram protocol network, a generic routing encapsulation network, or the like. Therefore, efficiency of forwarding the modified packet is further improved.", + "p0037": "The following clearly describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are merely a part rather than all of the embodiments of the present invention. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.", + "p0038": "FIG. 2 is a flowchart of a method for processing a modified packet according to an embodiment of the present invention. The method shown in FIG. 2 includes the following content.", + "p0039": "S 201 . A first routing device obtains a first packet sent by a second routing device, where the first packet carries a first correspondence, the first correspondence includes a correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and the second routing device is connected to the first routing device.", + "p0040": "It should be noted herein that, for such a connection relationship between an SF and a routing device, reference may be made to a connection relationship shown in FIG. 1 . As shown in FIG. 1 , an SF 1 , an SF 2 , and an SF 3 are connected to a routing device 1 , and an SF 4 and an SF 5 are connected to a routing device 2 . Such a connection relationship between an SF and a routing device cannot be understood as a connection between the SF 1 and the routing device 2 . Such a connection relationship between an SF and a routing device may also be referred to as attachment in the aft. When an SF is connected to the second routing device, the second routing device may obtain, by using a discovery protocol packet or a handshake protocol packet interacted between the second routing device and the SF, a correspondence between an ID of the SF and interface information of an interface that is of the second routing device and by using which the second routing device is connected to the SF. Optionally, the second routing device usually stores, in a local SF attachment table, the correspondence between the ID of the SF and the interface information of the interface of the second routing device for connecting to the SF. It is assumed that the second routing device is connected to totally three SFs, and IDs of the three SFs are respectively an SF 1 , an SF 2 , and an SF 3 . The local SF attachment table of the second routing device may be shown in Table 2.", + "p0041": "TABLE 2 Interface information of an interface of a ID of an SF routing device for connecting to an SF SF 1 Interface 1 SF 2 Interface 2 SF 3 Interface 3", + "p0042": "When the second routing device is to send the first packet, the second routing device may extract the IDs of the three SFs connected to the second routing device from the local SF attachment table, determine the first correspondence between the IDs of the three SFs connected to the second routing device and the network address information of the second routing device, and add the first correspondence to the first packet. The network address information of the second routing device may be a network address used to identify the second routing device, or may be a network address used to identify a network interface of the second routing device for connecting to another routing device. The network address may be an IP address, a MAC address, or the like, or may be a tunnel port address of a forwarding tunnel established between the second routing device and the first routing device, such as a GRE tunnel address or a UDP port address.", + "p0043": "S 202 . The first routing device creates a forwarding entry, where the forwarding entry includes the first correspondence, and the network address information of the second routing device in the forwarding entry indicates address information of a next-hop routing device of the first routing device on a service chain.", + "p0044": "It is assumed that the second routing device is connected to totally three SFs, and IDs of the three SFs are respectively an SF 1 , an SF 2 , and an SF 3 , the forwarding entry created by the first routing device may be shown in Table 3.", + "p0045": "TABLE 3 Network address information of a ID of an SF routing device connected to an SF SF 1 Network address information of a second routing device SF 2 Network address information of a second routing device SF 3 Network address information of a second routing device", + "p0046": "In S 202 , the first routing device creates the forwarding entry, and the forwarding entry may be understood as being for the second routing device. When the first routing device needs to forward a modified packet to the second routing device, the first routing device may determine the network address information of the second routing device by querying the forwarding entry. The following further describes the forwarding entry by describing how the first routing device forwards the modified packet by using the forwarding entry.", + "p0047": "S 203 . The first routing device receives a modified packet, and determines an ID of a destination SF of the modified packet.", + "p0048": "It should be noted herein that the modified packet mentioned in this embodiment of the present invention mainly refers to a packet to which a service chain header is added and that is forwarded on the service chain.", + "p0049": "The following describes how the first routing device determines the ID of the destination SF of the modified packet by using several cases of forwarding the modified packet. FIG. 3 is a schematic diagram of a network for forwarding a modified packet according to an embodiment of the present invention. As shown in FIG. 3 , a classifier, a routing device A, a routing device B, and a routing device C are included. Each routing device is connected to two SFs, and total six SFs are respectively an SF 1 to an SF 6 .", + "p0050": "A first case of forwarding the modified packet:", + "p0051": "It is assumed that a path that the modified packet sent by the classifier needs to pass through includes: the routing device A\u2192the SF 1 \u2192the routing device A\u2192the routing device B\u2192the SF 3 \u2192the routing device B. It is assumed that the routing device A is the first routing device, and the routing device B is the second routing device.", + "p0052": "When the modified packet is sent to the routing device A from the classifier, the service chain header of the modified packet includes a service chain ID and an ID of the SF 1 . The routing device A further has an SF forwarding entry that is for the service chain ID, and the SF forwarding entry includes a correspondence between a service chain ID and an SF. It is assumed that the service chain ID is 100, and the SF forwarding entry may be shown in Table 4a. When the ID of the destination SF is the SF 1 , an ID of a corresponding source SF is not shown in Table 4a. The SF 1 is a first SF on the path that the modified packet needs to pass through. Therefore, the modified packet has not been processed by an SF before reaching the SF 1 , and there is no ID of a source SF. In another possible implementation manner, an ID of a corresponding source SF when the ID of the destination SF is the SF 1 may also be set as a preset specific identifier for representation.", + "p0053": "TABLE 4a Service ID of a ID of a chain ID source SF destination SF 100 SF 1 100 SF 1 SF 3", + "p0054": "The routing device A sends the modified packet to the SF 1 , and the routing device A receives the modified packet returned from the SF 1 . In this case, the service chain header of the modified packet includes the service chain ID: 100 and the ID of the source SF: the SF 1 . The first routing device determines, by querying the SF forwarding entry shown in Table 4a, that an ID of a next-hop SF, that is, the ID of the destination SF, is the SF 3 .", + "p0055": "A second case of forwarding the modified packet:", + "p0056": "It is assumed that a path that the modified packet sent by the classifier needs to pass through includes: the routing device A\u2192the routing device B\u2192the SF 3 \u2192the routing device B. It is assumed that the routing device A is the first routing device, and the routing device B is the second routing device.", + "p0057": "The routing device A has an SF forwarding entry. It is assumed that a service chain ID is 110, and the SF forwarding entry is shown in Table 4b. When the ID of the destination SF is the SF 3 , an ID of a corresponding source SF is not shown in Table 4b. The SF 3 is a first SF on the path that the modified packet needs to pass through. Therefore, the modified packet has not been processed by an SF before reaching the SF 3 , and there is no ID of a source SF.", + "p0058": "TABLE 4b Service ID of a ID of a chain ID source SF destination SF 110 SF 3", + "p0059": "When the modified packet is sent to the routing device A from the classifier, the service chain header of the modified packet includes the service chain ID and an ID of the SF 3 . The routing device A may directly determine the ID of the destination SF of the modified packet by using Table 4b and the ID of the SF 3 included in the service chain header.", + "p0060": "It should further be noted that the network topology structure shown in FIG. 3 is merely an example. The network topology structure is not limited to only three routing devices, and a connection relationship between routing devices is not limited to a chain connection shown in FIG. 3 , and may also be a mesh structure or the like. The present invention imposes no limitation on a quantity of routing devices in a network topology structure formed by the routing devices and a connection relationship between the routing devices.", + "p0061": "S 204 . The first routing device queries the forwarding entry according to the ID of the destination SF, obtains the network address information of the second routing device from the first correspondence that matches the ID of the destination SF, and determines that an SF identified by the ID of the destination SF is the SF connected to the second routing device.", + "p0062": "FIG. 3 is used as an example. The routing device A queries the forwarding entry according to the ID of the destination SF: the SF 3 , and the forwarding entry is shown in Table 5:", + "p0063": "TABLE 5 Network address information of a ID of an SF routing device connected to an SF SF 3 Network address information of a routing device B SF 4 Network address information of a routing device B", + "p0064": "The network address information of the routing device B is obtained from the first correspondence that matches the SF 3 , and accordingly, the SF 3 is an ID of one of SFs connected to the second routing device.", + "p0065": "S 205 . The first routing device forwards the modified packet to the second routing device according to the network address information of the second routing device.", + "p0066": "In the routing device, a service forwarding entity may perform a function of forwarding the modified packet on the service chain.", + "p0067": "It may be learned from the embodiment corresponding to FIG. 2 that a forwarding entry used for forwarding a modified packet between routing devices does not need to be additionally configured for and delivered to a first routing device by a controller. The first routing device may obtain a first correspondence by using a first packet sent by a second routing device connected to the first routing device, and create the forwarding entry that includes the first correspondence. Therefore, when the first routing device needs to forward, to the second routing device, the modified packet whose ID of a destination SF is an ID of an SF connected to the second routing device, the first routing device may query the forwarding entry, and obtain network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0068": "In addition to creating the forwarding entry for the second routing device according to the first packet sent by the second routing device, the first routing device may further send, by using a second packet, a second correspondence between an ID of an SF connected to the first routing device and network address information of the first routing device to a third routing device connected to the first routing device, so that the third routing device may create a forwarding entry for the first routing device according to the second correspondence in the second packet.", + "p0069": "Based on the embodiment shown in FIG. 2 , FIG. 4 is a flowchart of a method for creating a forwarding entry according to an embodiment of the present invention. As shown in FIG. 4 , the method includes the following steps.", + "p0070": "S 401 . The first routing device obtains a second correspondence, where the second correspondence is a correspondence between network address information of the first routing device and an ID of an SF connected to the first routing device.", + "p0071": "Similar to the description in S 201 , when being connected to an SF, the first routing device may further obtain a correspondence between an ID of the SF and port information of a port that is of the first routing device and by using which the first routing device is connected to the SF. Optionally, the first routing device may further store a local SF attachment table similar to Table 2. When the first routing device is to send the second packet, the first routing device may extract, from the local SF attachment table, the ID of the SF connected to the first routing device, and determine the second correspondence between the ID of the SF connected to the first routing device and the network address information of the first routing device.", + "p0072": "S 402 . The first routing device sends a second packet to a third routing device connected to the first routing device, where the second packet carries the second correspondence.", + "p0073": "When the third routing device receives the second packet, the third routing device may create a forwarding entry that includes the second correspondence. When the third routing device needs to forward the modified packet to the first routing device, the third routing device may determine the network address information of the first routing device by querying the forwarding entry that includes the second correspondence.", + "p0074": "It should be noted that in S 402 , whether the second routing device and the third routing device are same routing devices is not limited. Therefore, when the second routing device and the third routing device are same routing devices, S 402 may be considered as: the first routing device sends, to the second routing device, the second packet that carries the second correspondence. The second routing device may create a forwarding entry that includes the second correspondence. When the third routing device, that is, the second routing device, needs to forward the modified packet on the service chain to the first routing device, the third routing device may determine the network address information of the first routing device by querying the forwarding entry that includes the second correspondence.", + "p0075": "When the third routing device and the second routing device are different devices, the first routing device may further send the first correspondence obtained from the second routing device to the third routing device by using a third packet, so that the third routing device may further create a forwarding entry for the second routing device. In this case, the second packet and the third packet may be same packets, and the third packet may carry both the first correspondence and the second correspondence.", + "p0076": "That is, optionally, the first routing device sends a third packet to the third routing device connected to the first routing device. The third packet carries the first correspondence, and the third routing device is different from the second routing device.", + "p0077": "It should be noted herein that the first correspondence is the correspondence between the network address information of the second routing device and the ID of the SF connected to the second routing device. If the third packet that includes the first correspondence is further sent to the second routing device, a packet broadcast storm may be caused in a network topology formed by the routing devices. Therefore, when the first routing device sends the third packet to the third routing device, the third routing device is not the second routing device that already has the first correspondence.", + "p0078": "FIG. 3 is used as an example. It is assumed that the routing device A is the third routing device, the routing device B is the first routing device, and the routing device C is the second routing device. The routing device B obtains the first packet that is sent by the routing device C connected to the routing device B and that carries the first correspondence. A forwarding entry that is accordingly created by the routing device B for the routing device C is shown in Table 6:", + "p0079": "TABLE 6 Network address information of a IB of an SF routing device connected to an SF SF 5 Network address information of a routing device C SF 6 Network address information of a routing device C", + "p0080": "The routing device B sends, to the connected routing device A, the second packet that carries the second correspondence, so that the routing device A creates a forwarding entry for the routing device B. The forwarding entry for the routing device B may be shown in Table 5. The routing device B may further send, to the routing device A, the third packet that carries the first correspondence, so that the routing device A creates a forwarding entry for the routing device C. The forwarding entry for the routing device C may be shown in Table 6.", + "p0081": "It is assumed that a path that the modified packet sent by the classifier needs to pass through includes: the routing device A\u2192the SF 1 \u2192the routing device A\u2192the routing device B\u2192the routing device C\u2192the SF 5 \u2192the routing device C.", + "p0082": "When the modified packet is sent to the routing device A from the classifier, the service chain header of the modified packet includes the service chain ID and the ID of the SF 1 . It is assumed that the service chain ID is 120, the SF forwarding entry that is on the routing device A and that is for the service chain ID may be shown in Table 7:", + "p0083": "TABLE 7 Service ID of a ID of a chain ID source SF destination SF 120 SF 1 120 SF 1 SF 5", + "p0084": "The routing device A sends the modified packet to the SF 1 , and the routing device A receives the modified packet returned from the SF 1 . In this case, the service chain header of the modified packet includes the service chain ID: 120 and an ID of a source SF: the SF 1 . The first routing device determines, by querying the SF forwarding entry shown in Table 7, that an ID of a next-hop SF, that is, the ID of the destination SF, is the SF 5 . The routing device A determines, by querying Table 6, network address information of the routing device C connected to the SF 5 , and accordingly determines that the ID of the destination SF of the modified packet is an ID of an SF connected to the routing device C . The routing device A may send the modified packet to the routing device C in an IP routing manner according to a connection relationship.", + "p0085": "In a manner of diffusing, between routing devices, a correspondence between network address information of a routing device and an ID of an SF connected to the routing device, the routing device may not only obtain a correspondence between network address information of a directly connected routing device and an ID of an SF connected to the directly connected routing device, but also obtain a correspondence between network address information of another routing device in a network topology and an ID of an SF connected to the another routing device, and accordingly create a forwarding entry for the another routing device. Therefore, compatibility of the routing device in the network topology and a capability of forwarding a modified packet are improved.", + "p0086": "The following describes types of the first packet, the second packet, and the third packet. To save a network resource, the first packet, the second packet, and the third packet may be control protocol packets between routing devices. The first packet is used as an example for description. For example, when a control protocol run by the first router and the second router is the Border Gateway Protocol , the first packet may be a BGP packet. A multi-protocol network layer reachability information part of the BGP packet may be used to carry the first correspondence. A format of the MP_REACH_NLRI part used to carry the first correspondence may be shown in Table 8:", + "p0087": "TABLE 8 Network address information of a second routing device ID list of an SF", + "p0088": "The ID list of an SF includes an ID or IDs of one or more SFs that is or are connected to the second routing device. When there are IDs of multiple SFs: an SF 1 to an SF n, a specific format of the MP_REACH_NLRI part used to carry the first correspondence may be shown in Table 8a or Table 8b:", + "p0089": "TABLE 8a Network address information of a second routing device SF 1 SF 2 . . . SF n", + "p0090": "TABLE 8b Network address information of a second routing device SF 1 SF 2 . . . SF n", + "p0091": "For example, when the control protocol run by the first router and the second router is the Label Distribution Protocol , the first correspondence may be carried by extending and defining a new type-length-value type, or the first correspondence may be carried by extending a new forwarding equivalence class based on an existing LDP mapping message.", + "p0092": "To further reduce volumes of data carried in the first packet, the second packet, and the third packet, the IDs of the SFs may further be aggregated. The first packet is used as an example for description. For example, in some cases, multiple SFs connected to the second routing device are separately an SF 1 , an SF 2 , and an SF 3 , and IDs of the three SFs are defined as: an ID of the SF 1 is 0000 0001, where the first four digits 0000 are a prefix, and the last four digits 0001 are a suffix; an ID of the SF 2 is 0000 0002, where the first four digits 0000 are a prefix, and the last four digits 0002 are a suffix; and an ID of the SF 3 is 0000 0003, where the first four digits 0000 are a prefix, and the last four digits 0003 are a suffix. Prefixes of the IDs of the SFs connected to the second routing device are the same, and prefixes of IDs of SFs connected to the first routing device are different. The three SFs have same prefixes. Therefore, the second routing device may represent the ID of the SF in the first correspondence by using a prefix of the ID of the SF, to reduce the volume of data carried in the first packet.", + "p0093": "In addition to adding the first correspondence to the first packet, the second routing device may further carry a network type of a network of the second routing device, such as a GRE network. In this way, when forwarding the modified packet on the service chain to the second routing device, the first routing device may further select a corresponding forwarding tunnel according to the network type of the network of the second routing device, to perform forwarding.", + "p0094": "Optionally, based on the embodiment shown in FIG. 2 or FIG. 4 , FIG. 5 is a flowchart of a method for creating a forwarding entry according to an embodiment of the present invention. As shown in FIG. 5 , the method includes the following steps.", + "p0095": "S 501 . The first packet obtained by the first routing device further carries a third correspondence, where the third correspondence includes a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and the network type includes a UDP network or a GRE network.", + "p0096": "S 502 . The forwarding entry created by the first routing device further includes the third correspondence.", + "p0097": "It should be noted that the second packet may further carry a correspondence between the network address information of the first routing device and a network type of a network of the first routing device, and the third packet may also carry the correspondence between the network address information of the first routing device and the network type of the network of the first routing device. The present invention sets no limitation thereto.", + "p0098": "FIG. 3 is used as an example for description. The routing device A is the first routing device, and the routing device B is the second routing device.", + "p0099": "The routing device A obtains the first packet that is sent by the routing device B and that carries the first correspondence and the third correspondence. It is assumed that a network type of a network of the routing device B is a GRE network, the forwarding entry created by the routing device A according to the first correspondence and the third correspondence may be shown in Table 9:", + "p0100": "TABLE 9 Network address information Network type of a network ID of of a routing device of a routing device an SF connected to an SF connected to an SF SF 3 Network address information GRE network of a routing device B SF 4 Network address information GRE network of a routing device B", + "p0101": "In this way, when the routing device A needs to forward the modified packet on the service chain to the routing device B, for example, when the ID of the destination SF of the modified packet is the SF 3 , by querying Table 9, the routing device A determines network address information of the routing device B connected to the SF 3 and determines that a network of the routing device B is a GRE network. The routing device A may forward the modified packet to the routing device B by using a forwarding tunnel corresponding to the GRE network.", + "p0102": "That is, based on the embodiment shown in FIG. 5 , optionally, that the first routing device forwards the modified packet to the second routing device according to the network address information of the second routing device includes:", + "p0103": "forwarding, by the first routing device, the modified packet to the second routing device according to the network address information of the second routing device by using a forwarding tunnel that is corresponding to the network type of the network of the second routing device, where the forwarding tunnel that is corresponding to the network type of the network of the second routing device is determined according to the third correspondence.", + "p0104": "It may be learned that a forwarding entry used for forwarding a modified packet between routing devices does not need to be additionally configured for and delivered to a first routing device by a controller. The first routing device may obtain a first packet sent by a second routing device connected to the first routing device, obtain a first correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and create the forwarding entry that includes the first correspondence, where the first correspondence is carried in the first packet. Therefore, when the first routing device needs to forward, to the second routing device, the modified packet whose ID of a destination SF is the ID of the SF connected to the second routing device, the first routing device may query the forwarding entry, and obtain the network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0105": "In a manner of diffusing, between routing devices, a correspondence between an ID of an SF and a routing device connected to the SF, a routing device may not only obtain a correspondence between network address information of a connected routing device and an ID of an SF connected to the connected routing device, but also obtain a correspondence between network address information of another connected routing device in a network topology and an ID of an SF connected to the another routing device, and accordingly create a forwarding entry for the another routing device. Therefore, compatibility of the routing device in the network topology and a capability of forwarding a modified packet are improved.", + "p0106": "In addition, the first packet received by the first routing device may further include a third correspondence between the network address information of the second routing device and a network type of the second routing device, so that the first routing device may select a corresponding forwarding tunnel to forward the modified packet to the second routing device, and efficiency of forwarding the modified packet is further improved.", + "p0107": "FIG. 6 is a structural diagram of an apparatus for processing a modified packet according to an embodiment of the present invention. As shown in FIG. 6 , an apparatus 600 for processing a modified packet includes a receiving unit 601 , a processing unit 602 , and a sending unit 603 .", + "p0108": "The receiving unit 601 is configured to obtain a first packet sent by a second routing device, where the first packet carries a first correspondence, the first correspondence includes a correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and the second routing device is connected to the apparatus 600 for processing a modified packet.", + "p0109": "It should be noted herein that, for such a connection relationship between an SF and a routing device, reference may be made to a connection relationship shown in FIG. 1 . As shown in FIG. 1 , an SF 1 , an SF 2 , and an SF 3 are connected to a routing device 1 , and an SF 4 and an SF 5 are connected to a routing device 2 . Such a connection relationship between an SF and a routing device cannot be understood as a connection between the SF 1 and the routing device 2 . Such a connection relationship between an SF and a routing device may also be referred to as attachment in the aft. When an SF is connected to the second routing device, the second routing device may obtain, by using a discovery protocol packet or a handshake protocol packet interacted between the second routing device and the SF, a correspondence between an ID of the SF and interface information of an interface that is of the second routing device and by using which the second routing device is connected to the SF. Optionally, the second routing device usually stores, in a local SF attachment table, the correspondence between the ID of the SF and the interface information of the interface of the second routing device for connecting to the SF. It is assumed that the second routing device is connected to totally three SFs, and IDs of the three SFs are respectively an SF 1 , an SF 2 , and an SF 3 , the local SF attachment table of the second routing device may be shown in Table 2.", + "p0110": "When the second routing device is to send the first packet, the second routing device may extract the IDs of the three SFs connected to the second routing device from the local SF attachment table, determine the first correspondence between the IDs of the three SFs connected to the second routing device and the network address information of the second routing device, and add the first correspondence to the first packet. The network address information of the second routing device may be a network address used to identify the second routing device, or may be a network address used to identify a network interface of the second routing device for connecting to another routing device. The network address may be an IP address, a MAC address, or the like, or may be a tunnel port address of a forwarding tunnel established between the second routing device and the apparatus 600 for processing a modified packet, such as a GRE tunnel address or a UDP port address.", + "p0111": "The processing unit 602 is configured to create a forwarding entry, where the forwarding entry includes the first correspondence, and the network address information of the second routing device in the forwarding entry indicates address information of a next-hop routing device, on a service chain, of the apparatus 600 for processing a modified packet.", + "p0112": "It is assumed that the second routing device is connected to totally three SFs, and IDs of the three SFs are respectively an SF 1 , an SF 2 , and an SF 3 , the forwarding entry created by the apparatus 600 for processing a modified packet may be shown in Table 3.", + "p0113": "The processing unit 602 creates the forwarding entry, and the forwarding entry may be understood as being for the second routing device. When the sending unit 603 needs to forward the modified packet to the second routing device, the network address information of the second routing device may be determined by querying the forwarding entry. The following further describes the forwarding entry by describing how to forward the modified packet by using the forwarding entry.", + "p0114": "The receiving unit 601 is further configured to receive the modified packet, and determine an ID of a destination SF of the modified packet.", + "p0115": "Description of how the receiving unit 601 determines the ID of the destination SF of the modified packet is similar to description in S 203 in the embodiment corresponding to FIG. 2 , and is not described herein again.", + "p0116": "The processing unit 602 is further configured to: query the forwarding entry according to the ID of the destination SF, obtain the network address information of the second routing device from the first correspondence that matches the ID of the destination SF, and determine that an SF identified by the ID of the destination SF is the SF connected to the second routing device.", + "p0117": "FIG. 3 is used as an example. The forwarding entry created by the routing device A may be shown in Table 5. It is assumed that when the ID of the destination SF of the modified packet is the SF 3 , by querying Table 5, network address information of a routing device B is obtained from the first correspondence that matches the SF 3 . Accordingly, the SF 3 is an ID of one of SFs connected to the second routing device.", + "p0118": "The sending unit 603 is configured to forward the modified packet to the second routing device according to the network address information of the second routing device.", + "p0119": "In the routing device, an SFE may perform a function of forwarding the modified packet on the service chain.", + "p0120": "It may be learned from the embodiment corresponding to FIG. 6 that a forwarding entry used for forwarding a modified packet between routing devices does not need to be additionally configured for and delivered to the apparatus 600 for processing a modified packet by a controller. The apparatus 600 for processing a modified packet may obtain the first correspondence by using a first packet sent by a second routing device connected to the apparatus 600 for processing a modified packet, and create the forwarding entry that includes the first correspondence. In this way, when the apparatus 600 for processing a modified packet needs to forward, to the second routing device, the modified packet whose ID of a destination SF is an ID of an SF connected to the second routing device, the apparatus 600 for processing a modified packet may query the forwarding entry, and obtain network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0121": "In addition to creating the forwarding entry for the second routing device according to the first packet sent by the second routing device, the apparatus 600 for processing a modified packet may further send, by using a second packet, a second correspondence between an ID of an SF connected to the apparatus 600 for processing a modified packet and network address information of the apparatus 600 for processing a modified packet to a third routing device connected to the apparatus 600 for processing a modified packet, so that the third routing device may create, according to the second correspondence in the second packet, a forwarding entry for the apparatus 600 for processing a modified packet.", + "p0122": "That is, optionally, the receiving unit 601 is further configured to obtain a second correspondence, where the second correspondence is a correspondence between network address information of the apparatus 600 for processing a modified packet and an ID of an SF connected to the apparatus 600 for processing a modified packet.", + "p0123": "When being connected to an SF, the receiving unit 601 may further obtain a correspondence between an ID of the SF and port information of a port that is of the apparatus 600 for processing a modified packet and by using which the apparatus 600 for processing a modified packet is connected to the SF. Optionally, the apparatus 600 for processing a modified packet may further store a local SF attachment table similar to Table 2. When the apparatus 600 for processing a modified packet is to send the second packet, the receiving unit 601 may extract, from the local SF attachment table, the ID of the SF connected to the apparatus 600 for processing a modified packet, and determine the second correspondence between the ID of the SF connected to the apparatus 600 for processing a modified packet and the network address information of the apparatus 600 for processing a modified packet.", + "p0124": "The sending unit 603 is further configured to send a second packet to a third routing device connected to the apparatus 600 for processing a modified packet, where the second packet carries the second correspondence.", + "p0125": "When the third routing device receives the second packet, the third routing device may create a forwarding entry that includes the second correspondence. When the third routing device needs to forward the modified packet to the apparatus 600 for processing a modified packet, the third routing device may determine, by querying the forwarding entry that includes the second correspondence, the network address information of the apparatus 600 for processing a modified packet.", + "p0126": "It should be noted that whether the second routing device and the third routing device are same routing devices is not limited herein. Therefore, when the second routing device and the third routing device are same routing devices, the sending unit 603 may be considered as: sending, to the second routing device, the second packet that carries the second correspondence. The second routing device may create a forwarding entry that includes the second correspondence. When the third routing device, that is, the second routing device, needs to forward the modified packet on the service chain to the apparatus 600 for processing a modified packet, the third routing device may determine, by querying the forwarding entry that includes the second correspondence, the network address information of the apparatus 600 for processing a modified packet.", + "p0127": "When the third routing device and the second routing device are different devices, the sending unit 603 may further send the first correspondence obtained from the second routing device to the third routing device by using a third packet, so that the third routing device may further create a forwarding entry for the second routing device. In this case, the second packet and the third packet may be same packets, and the third packet may carry both the first correspondence and the second correspondence.", + "p0128": "That is, optionally, the sending unit 603 is further configured to send the third packet to a third routing device connected to the apparatus 600 for processing a modified packet, where the third packet carries the first correspondence, and the third routing device is different from the second routing device.", + "p0129": "It should be noted herein that the first correspondence is the correspondence between the network address information of the second routing device and the ID of the SF connected to the second routing device. If the sending unit 603 further sends the third packet that includes the first correspondence to the second routing device, a packet broadcast storm may be caused in a network topology formed by the routing devices. Therefore, when the sending unit 603 sends the third packet to the third routing device, the third routing device is not the second routing device that already has the first correspondence.", + "p0130": "In addition to adding the first correspondence to the first packet, the second routing device may further carry a network type of a network of the second routing device, such as a GRE network. In this way, when forwarding the modified packet on the service chain to the second routing device, the apparatus 600 for processing a modified packet may further select a corresponding forwarding tunnel according to the network type of the network of the second routing device, to perform forwarding.", + "p0131": "That is, optionally, the first packet obtained by the receiving unit 601 further carries a third correspondence. The third correspondence includes a correspondence between the network address information of the second routing device and a network type of a network of the second routing device, and the network type includes a UDP network or a GRE network.", + "p0132": "The forwarding entry created by the processing unit 602 further includes the third correspondence.", + "p0133": "It should be noted that the second packet may further carry a correspondence between the network address information of the apparatus 600 for processing a modified packet and a network type of a network of the apparatus 600 for processing a modified packet. The third packet may also carry the correspondence between the network address information of the apparatus 600 for processing a modified packet and the network type of the network of the apparatus 600 for processing a modified packet. The present invention sets no limitation thereto.", + "p0134": "When the first packet obtained by the receiving unit 601 further carries the third correspondence, the sending unit 603 is further configured to forward the modified packet to the second routing device according to the network address information of the second routing device by using a forwarding tunnel that is corresponding to the network type of the network of the second routing device, where the forwarding tunnel that is corresponding to the network type of the network of the second routing device is determined according to the third correspondence.", + "p0135": "It may be learned that a forwarding entry used for forwarding a modified packet between routing devices does not need to be additionally configured for and delivered to the apparatus 600 for processing a modified packet by a controller. The apparatus 600 for processing a modified packet may obtain a first packet sent by a second routing device connected to the apparatus 600 for processing a modified packet, obtain a first correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and create the forwarding entry that includes the first correspondence, where the first correspondence is carried in the first packet. In this way, when the apparatus 600 for processing a modified packet needs to forward, to the second routing device, the modified packet whose ID of a destination SF is the ID of the SF connected to the second routing device, the apparatus 600 for processing a modified packet may query the forwarding entry, and obtain the network address information of the second routing device by matching the first correspondence, to complete forwarding. Therefore, a network resource is saved, and a process of creating the forwarding entry is simplified.", + "p0136": "In a manner of diffusing, between routing devices, a correspondence between an ID of an SF and a routing device connected to the SF, a routing device may not only obtain a correspondence between network address information of a connected routing device and an ID of an SF connected to the connected routing device, but also obtain a correspondence between network address information of another connected routing device in a network topology and an ID of an SF connected to the another routing device, and accordingly create a forwarding entry for the another routing device. Therefore, compatibility of the routing device in the network topology and a capability of forwarding a modified packet are improved.", + "p0137": "In addition, the first packet received by the apparatus 600 for processing a modified packet may further include a third correspondence between the network address information of the second routing device and a network type of the second routing device, so that the apparatus 600 for processing a modified packet may select a corresponding tunnel to forward the modified packet to the second routing device. Therefore, efficiency of forwarding the modified packet is further improved.", + "p0138": "Referring to FIG. 7 , FIG. 7 is a schematic diagram of a hardware structure of a first routing device according to an embodiment of the present invention. A first routing device 700 includes a memory 701 , a receiver 702 , a transmitter 703 , and a processor 704 that is connected to each of the memory 701 , the receiver 702 , and the transmitter 703 . The memory 701 is configured to store a group of program instruction, and the processor 704 is configured to invoke the program instruction stored in the memory 701 to perform the following operations. Triggering the receiver 702 to obtain a first packet sent by a second routing device, where the first packet carries a first correspondence, the first correspondence includes a correspondence between network address information of the second routing device and an ID of an SF connected to the second routing device, and the second routing device is connected to the first routing device. Creating a forwarding entry, where the forwarding entry includes the first correspondence, and the network address information of the second routing device in the forwarding entry indicates address information of a next-hop routing device of the first routing device on a service chain. Triggering the receiver 702 to receive a modified packet and determine an ID of a destination SF of the modified packet. Querying the forwarding entry according to the ID of the destination SF, obtaining the network address information of the second routing device from the first correspondence that matches the ID of the destination SF, and determining that an SF identified by the ID of the destination SF is the SF connected to the second routing device. Triggering the transmitter 703 to forward the modified packet to the second routing device according to the network address information of the second routing device.", + "p0139": "Optionally, the processor 704 may be a central processing unit , the memory 701 may be an internal memory of a random access memory type, the receiver 702 and the transmitter 703 may include a common physical interface, and the physical interface may be an Ethernet interface or an asynchronous transfer mode interface. The processor 704 , the transmitter 703 , the receiver 702 , and the memory 701 may be integrated into one or more independent circuits or one or more pieces of hardware, for example, an application-specific integrated circuit .", + "p0140": "The word \u201cfirst\u201d in a first routing device, a first packet, and a first correspondence that are mentioned in the embodiments of the present invention is merely used as a name identifier, and does not mean being the first in a sequence. For the words \u201csecond\u201d and \u201cthird\u201d, this rule also applies.", + "p0141": "It should be noted that persons of ordinary skill in the art may understand that all or a part of the processes of the methods in the embodiments may be implemented by a computer program instructing relevant hardware. The program may be stored in a computer readable storage medium. When the program runs, the processes of the method embodiments are performed. The foregoing storage medium may include: a magnetic disk, an optical disc, a read-only memory , a random access memory , or the like.", + "p0142": "The method and the apparatus for processing a modified packet provided in the present invention are described in detail above. The principle and implementation of the present invention are described herein by using specific embodiments. The description about the foregoing embodiments is merely provided to help understand the method and core ideas of the present invention. In addition, persons of ordinary skill in the art can make variations and modifications to the present invention in terms of the specific implementations and application scopes according to the ideas of the present invention. Therefore, the content of specification shall not be construed as a limit to the present invention.", + "features": { + "F1": "The method comprises obtaining, by a first routing device, a first packet sent by a second routing device, wherein the first packet carries a first correspondence between network address information of the second routing device and an identifier of a service function node.", + "F2": "The method includes storing, by the first routing device, the first correspondence.", + "F3": "The method involves receiving, by the first routing device, a second packet carrying a service chain ID and an ID of a source SF node.", + "F4": "The method includes determining, by the first routing device, an ID of the SF node to process the second packet according to the service chain ID, the ID of the source SF node, and a correspondence associated with them.", + "F5": "The method involves forwarding, by the first routing device, the second packet to the second routing device according to the first correspondence.", + "F6": "The method further comprises obtaining, by the first routing device, a second correspondence between network address information of the first routing device and an ID of an SF node connected to it.", + "F7": "The method includes sending, by the first routing device, a third packet to a third routing device connected to the first routing device, wherein the third packet carries the second correspondence.", + "F8": "The method involves sending, by the first routing device, a fourth packet to the third routing device, wherein the fourth packet carries the first correspondence, with the third routing device being different from the second routing device.", + "F9": "The first packet obtained by the first routing device further carries a third correspondence between the network address information of the second routing device and a network type, which can be a user datagram protocol network or a generic routing encapsulation network.", + "F10": "The forwarding of the second packet comprises using a forwarding tunnel that corresponds to the network type of the network of the second routing device, where the tunnel is determined according to the third correspondence.", + "F11": "The method specifies that the SF node is not connected to the first routing device.", + "F12": "An apparatus comprises a non-transitory computer readable medium and a processor connected to it, where the medium stores a program executed by the processor to obtain a first packet carrying a correspondence between network address information of a second routing device and an identifier of a service function node.", + "F13": "The apparatus stores the first correspondence.", + "F14": "The apparatus receives a second packet carrying a service chain ID and an ID of a source SF node.", + "F15": "The apparatus determines an ID of the SF node to process the second packet according to the service chain ID, the ID of the source SF node, and a correspondence associated with them.", + "F16": "The apparatus forwards the second packet to the second routing device according to the first correspondence.", + "F17": "The program includes instructions to obtain a second correspondence between network address information of the apparatus and an ID of an SF node connected to it.", + "F18": "The program includes instructions to send a third packet to a third routing device connected to the apparatus, wherein the third packet carries the second correspondence.", + "F19": "The program includes instructions to send a fourth packet to the third routing device, wherein the fourth packet carries the first correspondence, with the third routing device being different from the second routing device.", + "F20": "The first packet carries a third correspondence between the network address information of the second routing device and a network type, which can be a user datagram protocol network or a generic routing encapsulation network.", + "F21": "The program includes instructions to forward the second packet to the second routing device using a forwarding tunnel corresponding to the network type of the network of the second routing device, determined according to the third correspondence.", + "F22": "The apparatus specifies that the SF node is not connected to the first routing device." + } + } + }, + { + "FAN": "68249923", + "CPC": [ + "G06F-003/038", + "G06F-003/16/7", + "G06F-016/00", + "G06F-016/90332", + "G06F-017/30", + "G06F-2203/0381", + "G10L-015/22", + "G10L-025/48", + "G10L-2015/088", + "G10L-2015/226", + "H04H-060/33", + "H04N-021/25", + "H04N-021/45" + ], + "Content": { + "title": "Server for processing speech, control method thereof, image processing apparatus, and control method thereof", + "pa01": "A system includes a server and an image processing apparatus, and the server is provided that includes a communication interface, a storage, and a processor. The communication interface is configured to communicate with the image processing apparatus. The storage is configured to store data. The processor may provide a result of processing a first event that includes a speech of a user to the image processing apparatus in response to the first event being received from the image processing apparatus, store a record of the first event in the storage according to processing of the first event, determine a relation between the first and second events that includes a user input by a non-speech method in response to the second event being received from the image processing apparatus, and process the second event based on the record of the first event stored in the storage in response to the relation.", + "c-en-0001": "A system comprising: a server; and at least one image processing apparatus, wherein the server comprises: a communicator configured to communicate with the at least one image processing apparatus; a processor configured to: based on an input of a speech of a user being received from the at least one image processing apparatus, control the communicator to: transmit, to the at least one image processing apparatus, a plurality of retrieval results corresponding to the input of the speech of the user, wherein the at least one image processing apparatus comprises: a display; a user interface comprising: a speech input interface configured to receive the input of a speech of a user; and a non-speech input interface configured to receive a non-speech user input; a communicator configured to communicate with the server; and a processor configured to: based on the input of the speech being received through the speech input interface, control the communicator of the at least one image processing apparatus to transmit the input of the speech to the server; control the display to display the plurality of retrieval results to be selected, based on the plurality of retrieval results corresponding to the input of the speech being received from the server; based on the non-speech user input being received through the non-speech input interface, identify whether the non-speech user input is related to the input of the speech of the user; based on the non-speech user input not being related to the input of the speech of the user, perform an operation corresponding to the non-speech user input, independently of the input of the speech; and based on the non-speech user input being related to the input of the speech of the user, perform an operation on one of the plurality of retrieval results selected by the non-speech user input, based on a result of processing the input of the speech.", + "c-en-0002": "The system of claim 1 , wherein the processor of the server is further configured to: perform retrieval from a database based on the input of the speech when the input of the speech is received, and control the communicator of the server to transmit, to the at least one image processing apparatus, one final result selected from among a plurality of retrieval results with respect to the input of the speech, based on the non-speech user input being received when the retrieval results are being displayed.", + "c-en-0003": "The system of claim 1 , wherein the processor of the server is further configured to: control the communicator of the server to transmit a request command to select one of the retrieval results when there is the plurality of retrieval results with respect to the input of the speech.", + "c-en-0004": "The system of claim 1 , wherein the processor of the at least one image processing apparatus is further configured to convert the non-speech user input into a virtual voice command such that the server recognizes the non-speech user input, and control the communicator to transmit the virtual voice command to the server.", + "c-en-0005": "A control method of a server, the control method comprising: receiving, from an image processing apparatus, an input of a speech of a user; transmitting, to the image processing apparatus, a plurality of retrieval results corresponding to the input of the speech to be displayed on the image processing apparatus; receiving, by the image processing apparatus, at least one image from the server the plurality of retrieval results corresponding to the input of the speech; displaying, on the image processing apparatus, the plurality of retrieval results to be selected on a display of the image processing apparatus; receiving a non-speech user input through a non-speech input interface of the image processing apparatus; based on the non-speech user input being received through the non-speech input interface, identifying, by the image processing apparatus, whether the non-speech user input is related to the input of the speech of the user; based on the non-speech user input not being related to the input of the speech of the user, performing, by the image processing apparatus, an operation corresponding to the non-speech user input, independently of the input of the speech; and based on the non-speech user input being related to the input of the speech of the user, performing, by the image processing apparatus, an operation on one of the plurality of retrieval results selected by the non-speech user input, based on a result of processing the input of the speech.", + "c-en-0006": "The control method of claim 5 , further comprising: performing retrieval from a database based on the input of the speech, and transmitting, to the image processing apparatus, one final result selected among a plurality of retrieval results with respect to the input of the speech, based on the non-speech user input being received when the retrieval results are being displayed.", + "c-en-0007": "The control method of claim 5 , wherein the receiving of the non-speech user input comprises: transmitting, to the image processing apparatus, a request command configured to select one of the retrieval results when there is the plurality of retrieval results with respect to the input of the speech; and receiving, from the image processing apparatus, the non-speech user input corresponding to the request command.", + "c-en-0008": "The control method of claim 5 , further comprising: converting, by the image processing apparatus, the non-speech user input into a virtual voice command such that the server recognizes the non-speech user input, and transmitting, by the image processing apparatus, the virtual voice command to the server.", + "c-en-0009": "An image processing apparatus comprising: a display; a user interface comprising: a speech input interface configured to receive an input of a speech of a user; and a non-speech input interface configured to receive a non-speech user input; a communicator configured to communicate with a server that performs retrieval processing corresponding to the input of the speech; and a processor configured to: based on the input of the speech being received through the speech input interface, control the communicator to transmit the input of the speech to the server; control the display to display a received plurality of retrieval results to be selected, based on the plurality of retrieval results corresponding to the input of the speech being received from the server; based on the non-speech user input being received through the non-speech input interface, identify whether the non-speech user input is related to the input of the speech of the user; based on the non-speech user input not being related to the input of the speech of the user, perform an operation corresponding to the non-speech user input, independently of the input of the speech; and based on the non-speech user input being related to the input of the speech of the user, perform an operation on one of the plurality of retrieval results is selected by the non-speech user input, based on a result of processing the input of the speech.", + "c-en-0010": "The image processing apparatus of claim 9 , wherein the processor is further configured to identify whether the non-speech user input gives an instruction to select one of the retrieval results based on the non-speech user input being received when the retrieval results are being displayed.", + "c-en-0011": "The image processing apparatus of claim 9 , wherein the processor is further configured to convert the non-speech user input into a virtual voice command such that the server recognizes the non-speech user input, and control the communicator to transmit the virtual voice command to the server.", + "c-en-0012": "A control method of an image processing apparatus, the control method comprising: receiving an input of a speech of a user; communicating with a server that performs retrieval processing on the input of the speech; transmitting the input of the speech to the server based on the input of the speech being received; receiving from the server a plurality of retrieval results on the input of the speech; displaying the plurality of retrieval results for selection on a display of the image processing apparatus; receiving a non-speech user input through a non-speech input interface of the image processing apparatus; based on the non-speech user input being received through the non-speech input interface, identifying whether the non-speech user input is related to the input of the speech of the user; based on the non-speech user input not being related to the input of the speech of the user, performing an operation corresponding to the non-speech user input, independently of the input of the speech; and based on the non-speech user input being related to the input of the speech of the user, performing an operation on one of the plurality of retrieval results selected by the non-speech user input, based on a result of processing the input of the speech.", + "c-en-0013": "The control method of claim 12 , further comprising: identifying whether the non-speech user input gives an instruction to select one of the retrieval results based on the non-speech user input being received when the retrieval results are being displayed.", + "c-en-0014": "The control method of claim 13 , further comprising: converting the non-speech user input into a virtual voice command that the server recognizes, and transmitting the virtual voice command to the server.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATION", + "p0002": "This application claims priority from Korean Patent Application No. 10-2013-0077189, filed on Jul. 2, 2013 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.", + "p0003": "BACKGROUND", + "p0004": "1. Field", + "p0005": "Apparatuses consistent with exemplary embodiments relate to a server configured to process an image signal to be displayed as an image in a system, a control method thereof, an image processing apparatus, and a control method thereof, more particularly to a server configured to recognize a voice command of a user to perform a function or operation corresponding to the voice command, a control method thereof, an image processing apparatus, and a control method thereof.", + "p0006": "2. Description of the Related Art", + "p0007": "An image processing apparatus processes an image signal and/or image data received from the outside according to various types of image processing processes. The image processing apparatus may display an image based on the processed image signal on its own display panel or output the processed image signal to a display apparatus, including a panel, to display an image based on the image signal. That is, the image processing apparatus may include any device capable of processing an image signal, regardless of whether it includes a panel to display an image. For example, an image processing apparatus having a display panel may be a TV, while an image processing apparatus having no display panel may be a set-top box.", + "p0008": "As a variety of additional and extended functions are being added to an image processing apparatus as technology develops, diverse configurations and methods of inputting user desired commands to the image processing apparatus in order to request or control these extended functions are also being developed. For example, when a user presses a key/button on a remote controller, the remote controller transmits a control signal to an image processing apparatus so that a user desired operation is performed. Currently, various configurations of controlling an image processing apparatus considering a user desire are proposed. For example, an image processing apparatus detects a motion or voice of a user and analyzes the detected data to perform a relevant operation.", + "p0009": "SUMMARY", + "p0010": "According to an aspect of an exemplary embodiment, there is provided a system including a server and at least one image processing apparatus, the server including a communication interface configured to communicate with the at least one image processing apparatus, a storage configured to store data, and a processor configured to provide a result of processing a first event that includes a speech of a user to the image processing apparatus in response to the first event being received from the image processing apparatus, store a record of the first event in the storage according to processing of the first event, determine a relation between the first event and a second event that includes a user input by a non-speech method in response to the second event being received from the image processing apparatus, and process the second event based on the record of the first event stored in the storage in response to the relation.", + "p0011": "The processor may determines whether to process the second event independently of the first event or to process the second event based on the result of processing the first event when the second event is received using the record of the first event stored in the storage.", + "p0012": "The processor may be further configured to perform retrieval from a preset database based on the first event when the first event is received, and transmit, to the image processing apparatus, one final result selected based on the second event from among a plurality of retrieval results with respect to the first event in response to the second event being received with the plurality of retrieval results being derived from the first event.", + "p0013": "The processor may be further configured to transmit, to the image processing apparatus, the plurality of retrieval results and a request command to select one of the retrieval results when there is the plurality of retrieval results with respect to the first event, and receive, from the image processing apparatus, the second event corresponding to the request command.", + "p0014": "The second event may be converted into a virtual voice command recognizable and received by the server.", + "p0015": "According to an aspect of another exemplary embodiment, there is provided a control method of a server, the control method including receiving, from an image processing apparatus, a first event based on a speech of a user, providing a result of processing the first event to the image processing apparatus and storing a record of the first event according to processing of the first event, receiving, from the image processing apparatus, a second event based on a user input by a non-speech method, determining a relation between the first event and the second event, and processing the second event based on a pre-stored record of the first event in response to the relation.", + "p0016": "The determining of the relation may include determining whether to process the second event independently of the first event or to process the second event based on the result of processing the first event using the pre-stored record stored in the storage.", + "p0017": "The providing of the result of processing the first event to the image processing apparatus may include performing retrieval from a preset database based on the first event. The processing of the second event based on the pre-stored record of the first event may include transmitting, to the image processing apparatus, one final result selected based on the second event among a plurality of retrieval results with respect to the first event in response to the second event being received with the plurality of retrieval results being derived from the first event.", + "p0018": "The receiving of the second event may include transmitting, to the image processing apparatus, the plurality of retrieval results and a request command configured to select one of the retrieval results when there is the plurality of retrieval results with respect to the first event and receiving, from the image processing apparatus, the second event corresponding to the request command.", + "p0019": "The second event may be converted into a virtual voice command recognizable and received by the server.", + "p0020": "According to an aspect of another exemplary embodiment, there is provided an image processing apparatus including a storage configured to store data, a user interface including a speech input interface configured to receive an input of a speech of a user, and a non-speech input interface configured to receive a non-speech user input, a processor configured to perform processing on an operation corresponding to a first event that corresponds to the input of the speech at the speech input interface, and a controller configured to store a record of generating and processing the first event in the storage, determine a relation between the first event and a second event, wherein the second event occurs at the non-speech input interface after the first event occurs, and control the processing of the second event based on the record of the first event stored in the storage in response to the relation.", + "p0021": "The controller may determine whether to process the second event independently of the first event or to process the second event based on a result of processing the first event when the second event occurs using the record of the first event stored in the storage.", + "p0022": "The controller may be further configured to display a plurality of retrieval results configured to be selected on a display, wherein the plurality of retrieval results with respect to the operation correspond to the first event, and determine whether the second event gives an instruction to select one of the retrieval results when the second event occurs when the retrieval results are being displayed.", + "p0023": "The image processing apparatus may further include a communication interface configured to communicate with a server that performs retrieval processing on a voice command corresponding to the speech, wherein the controller may transmit the first event to the server, receive a retrieval result with respect to the first event from the server, and display the retrieval result on the display.", + "p0024": "The controller may be further configured to convert the second event into a virtual voice command such that the server recognizes the second event, and transmits the virtual voice command to the server.", + "p0025": "According to an aspect of another exemplary embodiment, there is provided a control method of an image processing apparatus. The control method includes generating a first event based on an input of a speech of a user, wherein a speech input interface is configured to receive the input, and performing an operation corresponding to the first event, wherein the performing of the operation corresponding to the first event includes storing a record of generating and processing the first event in a storage, generating a second event based on a non-speech user input, wherein a non-speech input interface is configured to receive a non-speech user input, determining a relation between the first event and the second event, and processing the second event based on the record of the first event in response to the relation.", + "p0026": "The determining of the relation may include determining whether to process the second event independently of the first event or to process the second event based on a result of processing the first event using the record of the first event.", + "p0027": "The performing of the operation corresponding to the first event may further include displaying a plurality of retrieval results configured to be selected, wherein the plurality of retrieval results with respect to the operation correspond to the first event, and wherein the determining of the relation between the first event and the second event may include determining whether the second event gives an instruction to select one of the retrieval results.", + "p0028": "The image processing apparatus may be further configured to communicate with a server that performs retrieval processing on a voice command corresponding to the speech, transmit the first event to the server, and display a retrieval result with respect to the first event received from the server.", + "p0029": "The image processing apparatus may be further configured to convert the second event into a virtual voice command that the server recognizes, and transmit the virtual voice command to the server.", + "p0030": "According to an aspect of another exemplary embodiment, there is provided an image processing apparatus including a user interface including a speech input interface configured to receive an input of a speech of a user, and a non-speech input interface configured to receive a non-speech user input, a communication interface configured to communicate with a server that retrieves an operation corresponding to a voice command due to the speech, a processor configured to be provided, from the server, a result of retrieving a first event that corresponds to the input of the speech at the speech input interface, and configured to process the retrieval result, and a controller configured to convert a second event into a virtual voice command such that the server recognizes the second event, transmit the virtual voice command to the server so that the server selects one of a plurality of retrieval results based on the second event, and provide a final retrieval result when the second event gives an instruction to select one of the retrieval results based on the first event that is derived by the server.", + "p0031": "According to an aspect of another exemplary embodiment, there is provided a user input determination system including: an image processing apparatus including: a user interface configured to receive a first input and a second input from a user; a processor configured to process the first input and the second input into a first command and a second command, respectively; and a communication interface configured to transmit the first command and the second command; and a server including: a server communication interface configured to receive the first command and the second command; and a server processor configured to determine content to be displayed based on the first command and the second command.", + "p0032": "The processor of the image processing apparatus may be further configured to determine content to be displayed based on the first command and the second command.", + "p0033": "The first input may be an vocalization by the user; wherein the second input may be one from a group consisting of a visual gesture and a touch; wherein the first command may be a voice command understandable by the processor and the server processor; and wherein the second command may be a virtual voice command understandable by the processor and the server processor.", + "p0034": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0035": "The above and/or other aspects will become apparent and more readily appreciated from the following description of the exemplary embodiments, taken in conjunction with the accompanying drawings, in which:", + "p0036": "FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus included in a system according to an exemplary embodiment.", + "p0037": "FIG. 2 is a block diagram illustrating a configuration of a server included in the system according to an exemplary embodiment.", + "p0038": "FIGS. 3 and 4 are flowcharts illustrating a method where an interactive server provides a search result corresponding to an event to a display apparatus according to one or more exemplary embodiments.", + "p0039": "FIG. 5 is a block diagram illustrating a configuration of subdivisions by functions of a server processor similar to the interactive server of FIG. 2 according to an exemplary embodiment.", + "p0040": "FIGS. 6 and 7 illustrate data transmission relations between the display apparatus and the interactive server similar to the server of FIG. 2 and between components of the interactive server according to one or more exemplary embodiments.", + "p0041": "FIGS. 8 and 9 are flowcharts illustrating a control method of a display apparatus according to one or more exemplary embodiments.", + "p0042": "FIG. 10 is a block diagram illustrating a configuration of a processor of a display apparatus according to an exemplary embodiment.", + "p0043": "DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS", + "p0044": "Below, exemplary embodiments will be described in detail with reference to the accompanying drawings. The exemplary embodiments may be embodied in various forms without being limited to the exemplary embodiments set forth herein. Descriptions of well-known parts are omitted for clarity and conciseness, and like reference numerals refer to like elements throughout.", + "p0045": "FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus 100 included in a system according to an exemplary embodiment.", + "p0046": "The present exemplary embodiment will be illustrated with a display apparatus capable of autonomously displaying an image, for example, a TV, as the image processing apparatus 100 . However, the idea of the present embodiment may be also applied to an image processing apparatus which does not autonomously display an image but outputs image data or control information to another display apparatus, without being limited to the following description. That is, the idea of the present embodiment may be changed and modified variously and thus be applied to diverse types of devices.", + "p0047": "Referring to FIG. 1 , the image processing apparatus 100 , or display apparatus 100 , according to the present embodiment receives an image signal from an external image source. The display apparatus 100 may receive any image signal, which is not limited to a particular kind or particular characteristic. For example, the display apparatus 100 may receive a broadcast signal transmitted from a transmitter of a broadcasting station and tune the broadcast signal to display a broadcast image.", + "p0048": "The display apparatus 100 may include a communication interface 110 configured to communicate with an outside by transmitting and receiving data/signals, a processor 120 configured to process data received via the communication interface 110 according to a preset process, a display 130 configured to display image data as an image when data processed by the processor 120 is the image data, a user interface 140 configured to perform an operation input by a user, a storage 150 configured to store data/information, and a controller 160 configured to control whole operations of the display apparatus 100 .", + "p0049": "The communication interface 110 may transmit and receive data so that the display apparatus 100 may perform two-way communications with an external device, such as a server. The communication interface 110 may connect to the external device locally or via a wire-based or wireless wide area/local area network in accordance with a preset communication protocol.", + "p0050": "The communication interface 110 may be provided as a connection port for each device or an assembly of connection modules, and thus a protocol for connection or an external device as a target of connection is not limited to one kind or form. The communication interface 110 may be embedded in the display apparatus 100 , or the entire communication interface 110 or part of the communication interface 110 may be additionally installed in the display apparatus 100 as an add-on or dongle form.", + "p0051": "The communication interface 110 transmits and receives signals in accordance with a protocol designed for each connected device and thus may transmit and receive a signal based on an individual communication protocol for each connected device. In the case of image data, for example, the communication interface 110 may transmit and receive a radio frequency signal and various signals in accordance with composite video, component video, super video, Syndicat des Constructeurs d'Appareils Radior\u00e9cepteurs et T\u00e9l\u00e9viseurs , high definition multimedia interface , DisplayPort, unified display interface or wireless High Definition standards.", + "p0052": "The processor 120 may perform various processes on data or signals received by the communication interface 110 . For example, when image data is received by the communication interface 110 , the processor 120 may perform an image processing process on the image data and output the processed image data to the display 130 so that an image based on the image data is displayed on the display 130 . According to another example, when a broadcast signal is received by the communication interface 110 , the processor 120 may extract an image, audio data, and optional data from the broadcast signal tuned to a particular channel and may adjust the image to a preset resolution to be displayed on the display 130 .", + "p0053": "The processor 120 may perform any kind of image processing process, without being limited to, for example, decoding corresponding to an image format of image data, de-interlacing to convert interlaced image data into a progressive form, scaling to adjust image data to a preset resolution, noise reduction to improve image quality, detail enhancement, frame refresh rate conversion, or the like.", + "p0054": "The processor 120 may perform various kinds of processes depending on types and characteristics of data, and thus a process performed by the processor 120 is not limited to an image processing process. Further, data processed by the processor 120 is not limited to data received by the communication interface 110 . For example, when a user's speech is input through the user interface 140 , the processor 120 may process the speech according to a preset speech processing process.", + "p0055": "The processor 120 may be provided as an integrated multi-functional component, such as a system on chip , or as an image processing board formed by mounting separate chipsets which independently conduct individual processes on a printed circuit board and be embedded in the display apparatus 100 .", + "p0056": "The display 130 may display an image based on image signals/image data processed by the processor 120 . The display 130 may be configured in various display modes using liquid crystals, plasma, light emitting diodes, organic light emitting diodes, a surface conduction electron emitter, a carbon nano-tube, nano-crystals, or the like.", + "p0057": "The display 130 may further include an additional component depending on a display mode thereof. For example, in a display mode using liquid crystals, the display 130 includes a liquid crystal display panel, a backlight unit to provide light to the LCD panel, and a panel driving board driving the panel.", + "p0058": "The user interface 140 may transmit various preset control commands or information to the controller 160 based on a user manipulation or input. The user interface 140 informationalizes and transmits, to the controller 160 , various events that occur by a user based on the user's intent. For example, the events by the user that may be occurring may include diverse forms, for example, a manipulation, a speech, and a gesture.", + "p0059": "The user interface 140 is configured to detect input information in accordance with a method corresponding to a way in which the user inputs the information. To this end, the user interface 140 may be divided into an interactive input interface 141 and a non-interactive input interface 142 .", + "p0060": "The interactive input interface 141 is configured to receive a user's speech. That is, the interactive input interface 141 is provided as a microphone and detects various sounds generated in the surroundings of the display apparatus 100 . The interactive input interface 141 may generally detect a speech by a user and also detect sounds produced from various environmental factors other than the user.", + "p0061": "The non-interactive input interface 142 is configured to perform an input by a user other than a speech. In this case, the non-interactive input interface 142 may be realized in various forms, for example, a remote controller separate from the display apparatus 100 , a menu key or input panel installed on an outside of the display apparatus 100 , or a motion sensor or camera to detect a user's gesture.", + "p0062": "Alternatively, the non-interactive input interface 142 may be provided as a touchscreen installed on the display 130 . In this case, the user touches an input menu or UI image displayed on the display 130 to transmit a preset command or information to the controller 160 .", + "p0063": "The storage 150 may store various kinds of data according to control by the controller 160 . The storage 150 is configured as a nonvolatile memory, such as a flash memory and a hard disk drive, so as to save data regardless of supply of system power. The storage 150 is accessed by the controller 160 to read, record, revise, delete or update data stored therein.", + "p0064": "The controller 160 may be configured as a central processing unit and may control operations of whole components of the display apparatus 100 including the processor 120 as an event occurs. For example, in a case of a user speech event, when a user's speech is input through the interactive input interface 141 , the controller 160 controls the processor 120 to process the input speech to detect the word or phrase uttered by the user. Thus, when the user says a particular channel number that is detected, the controller 160 may control to switch to a broadcast image on the channel corresponding to the spoken channel number.", + "p0065": "According to one or more exemplary embodiments, processing the input speech may be carried out by the display apparatus 100 alone or be shared with an external device in consideration of different factors, such as load or throughput of a system, which will be described below.", + "p0066": "FIG. 2 is a block diagram illustrating configurations of servers 200 and 300 included in the system according to the present embodiment.", + "p0067": "As shown in FIG. 2 , the display apparatus 100 may connect to one or more servers 200 and 300 in order to communication through a network. The display apparatus 100 may include a speech-to-text server 200 configured to convert a speech of the user into a voice command and an interactive server 300 to analyze a voice command and determine an operation or action corresponding to the voice command. According to one or more exemplary embodiments, the terms of the servers 200 and 300 are used only to distinguish each other in view of functions thereof and do not limit to features in configuration.", + "p0068": "The STT server 200 may analyze a waveform of an audio signal and generate content of the audio signal into text. Thus, when an audio signal is received from the display apparatus 100 , the STT server 200 converts the audio signal into a voice command that includes text. The speech refers to an action done by the user, and the voice command refers to data in text form obtained from content of the speech.", + "p0069": "The interactive server 300 retrieves various operations for the display apparatus corresponding to the voice command. The interactive server 300 analyzes the voice command received from the display apparatus 100 and transmits a control signal to perform an operation corresponding to the voice command, or a result of search based on the voice command, to the display apparatus 100 according to an analysis result.", + "p0070": "The interactive server 300 may include a server communication interface 310 to communicate with the display apparatus 100 and a server processor 320 to process a voice command received through the server communication interface 310 .", + "p0071": "In the system that includes the display apparatus 100 , the STT server 200 , and the interactive server 300 , an operation performed when the user of the display apparatus 100 speaks will be described hereinafter.", + "p0072": "When speech of the user is input, the display apparatus 100 transmits an audio signal of the speech to the STT server 200 . The STT server 200 converts the audio signal received from the display apparatus into a voice command in a data form and transmits the voice command to the display apparatus 100 . The display apparatus 100 transmits the voice command received from the STT server 200 to the interactive server 300 .", + "p0073": "The interactive server 300 analyzes content of the voice command, derives a result corresponding to the voice command, and returns the result to the display apparatus 100 . For example, when the voice command includes content \u201cwho is the cast of \u2018Program ABC\u2019 this week?,\u201d the interactive server 300 conducts retrieval from an internal or external database with keywords \u201cthis week,\u201d \u201cProgram ABC,\u201d and \u201ccast\u201d and acquires a result.", + "p0074": "Here, the database is structured according to categories of keywords, and thus a desired search result may be obtained using a specific keyword. The structured database may be realized by various methods depending on design and construction of the database, which will not be described in detail.", + "p0075": "Further, when the interactive server 300 performs retrieval with one or more keywords derived from the voice command, a plurality of search results may be obtained. For example, in the case of voice command illustrated above, a show \u201cProgram ABC\u201d to be broadcast this week may be broadcast at different times or on different channels.", + "p0076": "In this case, the interactive server 300 may need to select one search result deemed proper for a user's intent from among the plurality of search results with respect to the voice command. The interactive server 300 may select one search result by two methods as follows.", + "p0077": "In a first method, one search result is selected based on a rule autonomously set by a client of the interactive server 300 , that is, the display apparatus 100 . For example, when the voice command inquires about air time for a particular program and there are a plurality of programs that satisfy the air time, a program to be broadcast sooner may be selected. In a second method, when the user of the display apparatus 100 speaks to select one search result, the interactive server 300 connects the speech to a previous speech record, thereby selecting a search result proper for the user's original intent.", + "p0078": "Because the system provides an environment for various input methods by the user, even though the display apparatus 100 supports a voice recognition input environment, the user may select a more convenient method for each situation among the various input methods. In the environment of the system for supporting multiple input modes, when a record of multiple inputs by the user is individually managed, there may be limitations in selecting or deriving a result based on the user's original intent. If the result based on the user's original intent is not provided when the user performs an input via a speech at first and then changes the input method to a different method, the user may find a user environment inconvenient.", + "p0079": "Thus, in an interactive environment of the system provided by the interactive server 300 and the display apparatus 100 , when a plurality of results are retrieved with respect to the speech of the user, and the user selects one search result through a non-speech input method, the interactive system may need to provide a search result proper for the user's original intent of the speech.", + "p0080": "Here, the system may provide integrated management of a record of multiple input modes. That is, the interactive server 300 manages an input by a speech of the user of the display apparatus 100 , an input by a gesture of the user and an input via manipulation of a remote controller by the user, and may also manage these inputs in connection with each other.", + "p0081": "In contrast, in the conventional system, the display apparatus 100 displays a plurality of search results received from the interactive server 300 according to the second method. Here, the user may perform an input via the non-interactive input interface 142 in a non-speech method, not via a speech. Further, the conventional interactive server 300 does not connect a record of speech inputs and a record of non-speech inputs for reference, and thus a result inappropriate for a user's intent is obtained.", + "p0082": "For example, suppose that the user says \u201cRecord Program ABC\u201d and a plurality of contents related to \u201cProgram ABC\u201d is displayed on the display apparatus 100 . In the conventional system, when the user speaks to select one content, one content is normally selected and recorded. However, when the user selects one content in a non-speech input method, content is selected only but is not recorded, because information based on a non-speech input by the user is not transmitted to the interactive server 300 so that the interactive server 300 does not perform an operation subsequent to the speech.", + "p0083": "In contrast, one or more exemplary embodiments suggest the following. Initially, an event that occurs by the interactive input interface 141 of the display apparatus 100 may be defined as a first event and an event occurs by the non-interactive input interface 142 may be defined as a second event. When the first event occurs from the display apparatus 100 , the interactive server 300 processes the first event and stores a record of the first event. Subsequently, when the second event occurs, the interactive server 300 processes the second event based on the stored record of the processed first event. Here, when the second event occurs, the display apparatus 100 converts the second event into a voice command and transmits the voice command to the interactive server 300 so that the interactive server 300 configured to recognize a voice command recognizes the second event.", + "p0084": "The interactive server 300 may refer to a record of previous events in determining an action on an event. The interactive server 300 may not separately manage events occurring by the interactive input interface 141 and by the non-interactive input interface 142 of the display apparatus 100 but manages the events in an integrated manner. Thus, the interactive server 300 may refer to a previous event in determining an action on a new event regardless of whether the previous event occurs by the interactive input interface 141 or by the non-interactive input interface 142 .", + "p0085": "For example, when a plurality of results are retrieved with respect to the first speech of the user, the interactive server 300 provides the plurality of search results to the user for selection. Here, although the user inputs an instruction to select one search result in a non-speech method, the interactive server 300 may associate a new event with a previous event even though the two events are input in different methods. Accordingly, a user desired search result is derived and provided to the user.", + "p0086": "FIGS. 3 and 4 are flowcharts illustrating a method where the interactive server 300 provides a search result corresponding to an event to the display apparatus 100 .", + "p0087": "As shown in FIG. 3 , in operation S 100 , the interactive server 300 receives a first event that may be, for example, a voice instruction by a speech from the display apparatus 100 .", + "p0088": "In operation S 110 , the interactive server 300 analyzes the first event to derive a search keyword.", + "p0089": "In operations S 120 , the interactive server 300 performs retrieval using the derived search keyword and stores a record of generating and processing the first event.", + "p0090": "In operation S 130 , the interactive server 300 determines whether there is a plurality of search results.", + "p0091": "When a single result is retrieved, the interactive server 300 transmits the search result to the display apparatus 100 and terminates in operation S 140 .", + "p0092": "When there is a plurality of search results, the interactive server 300 transmits the search results to the display apparatus 100 and requests the display apparatus 100 to select one of the search results in operation S 150 .", + "p0093": "As shown in FIG. 4 , the interactive server 300 receives a second event, which may include a selection instruction, from the display apparatus in operation S 160 . Here, the second event may occur by a user input to the display apparatus 100 in a non-speech method.", + "p0094": "In operation S 170 , the interactive server 300 determines relation between the first event and the second event, that is, whether to process the second event dependently from the first event, with reference to the pre-stored record of generating and processing the first event. For example, the interactive server 300 may determine whether the second event is a selection instruction with respect to the results retrieved by the first event.", + "p0095": "When there is no relation between the first event and the second event in operation S 180 , the interactive 300 analyzes and processes the second event separately from the first event in operation S 190 .", + "p0096": "However, when the second event is a selection instruction with respect to the results retrieved by the first event in operation S 180 , the interactive server 300 may select a search result based on the second event in operation S 200 . In this process, the interactive server 300 may perform additional retrieval if necessary.", + "p0097": "In operation S 210 , the interactive server 300 transmits a final search result according to the first event and the second event to the display apparatus 100 .", + "p0098": "Hereinafter, an internal configuration of the interactive server 300 will be described in detail with reference to FIG. 5 .", + "p0099": "FIG. 5 is a block diagram illustrating the configuration of subdivisions by functions of the server processor 320 of the interactive server 300 .", + "p0100": "As shown in FIG. 5 , the server processor 320 of the interactive server 300 may include a dialog engine 321 to analyze a voice command received through the server communication interface 310 and a content manager 322 to perform retrieval based on a search keyword transmitted from the dialog engine 321 .", + "p0101": "The dialog engine 321 includes a dialog engine manager 323 , a dialog agent 324 , an action agent 325 and a database 326 . The server processor 320 or the dialog engine 321 is functionally divided into the foregoing components for convenience but is not limited to the configuration mentioned above.", + "p0102": "Further, these components may be hardware elements, software elements or combinations thereof.", + "p0103": "The dialog engine manager 323 transmits an event in a form of a voice command transmitted to the dialog engine 321 to the dialog agent 324 . The dialog agent 324 analyzes content of the event, generates an analysis result into structured data, and returns the data to the dialog engine manager 323 .", + "p0104": "Here, the structured data generated by the dialog agent 324 is referred to as a dialog frame hereinafter. The dialog frame is data which includes the content of the voice command event and is obtained by structuring the voice command according to categories so that the dialog engine manager 323 , the action agent 325 and the content manager 322 may decode and refer to the command. Categories and structures may be determined variously depending on a design of the server processor 320 .", + "p0105": "The dialog frame may include a user's objective of the voice command, attributes needed to achieve the objective, and information on values of the attributes. The dialog frame will be described in detail.", + "p0106": "The dialog engine manager 323 transmits the dialog frame transmitted from the dialog agent 324 to the action agent 325 . The action agent 325 determines an action corresponding action agent 325 , and maybe from there to the content manager 322 , which may then return a determination results to the dialog engine manager 323 . That is, the action agent 325 determines an action corresponding to the event.", + "p0107": "When retrieval of information is needed in the determination process, the action agent 325 transmits a keyword for search extracted from the dialog frame to the content manager 322 to request retrieval of information. The content manager 322 performs retrieval with the transmitted keyword from various kinds of search databases inside or outside the interactive server 300 and returns a search result to the action agent 325 .", + "p0108": "When the action corresponding to the event or the search result is received from the action agent 325 , the dialog engine manager 323 transmits the received result to the server communication interface 310 to be transmitted to the display apparatus 100 .", + "p0109": "The database 326 includes whole information that the dialog engine manager 323 , the dialog agent 324 and the action agent 325 may refer to in the foregoing process. For example, the database 326 may include context information, event record information, and dialog model information that the dialog agent 324 may use to generate the dialog frame and action model information that the action agent 325 may then use to determine the action corresponding to the event.", + "p0110": "With this structure, an operation of the interactive server 300 returning a result corresponding to a voice command received from the display apparatus 100 to the display apparatus 100 will be described in detail.", + "p0111": "FIGS. 6 and 7 illustrate data transmission relations between the display apparatus 100 and the interactive server 300 and between the components of the interactive server 300 .", + "p0112": "As shown in FIG. 6 , when the user says \u201cwho is the cast of \u2018Program ABC\u2019 this week?\u201d, the display apparatus 100 acquires a voice command corresponding to the speech. The voice command corresponding to the speech is defined as a first event. The display apparatus 100 transmits the first event to the interactive server 300 .", + "p0113": "The dialog engine manager 323 transmits the first event received by the server communication interface 410 to the dialog agent 324 .", + "p0114": "The dialog agent 324 analyzes the first event to make a dialog frame corresponding to the first event. When the first event includes content \u201cwho is the cast of \u2018Program ABC\u2019 this week?,\u201d the dialog frame may be expressed as follows.", + "p0115": "Objective: To inquire about the cast of the program", + "p0116": "Title: Program ABC", + "p0117": "Time: This week", + "p0118": "In the dialog frame, \u201cObjective,\u201d \u201cTitle\u201d and \u201cTime\u201d are conditional categories of the dialog frame, and the respective conditional categories include attribute values of \u201cTo inquire about the case of the program,\u201d \u201cProgram ABC\u201d and \u201cThis week.\u201d The dialog agent 324 analyzes content of the event to derive a category of a condition which includes each keyword included in the first event and an attribute value by each category.", + "p0119": "When the dialog frame is received from the dialog agent 324 , the dialog engine manager 323 transmits the dialog frame to the action agent 325 .", + "p0120": "The action agent 325 makes the following keywords based on the transmitted dialog frame and transmits the keywords to the content manager 322 .", + "p0121": "Title: Program ABC", + "p0122": "Time: 2013/06/03\u02dc2013/06/09", + "p0123": "An attribute value of a time condition category may need to include a specific data for retrieval. To this end, the action agent 325 determines a specific date and time period this week as from current time on the basis of an attribute value of \u201cthis week\u201d in a time category of the dialog frame.", + "p0124": "The content manager 322 performs retrieval on a search service of various types of content and metadata using the keywords transmitted from the action agent 325 . A target of retrieval may be all database, search services and content services accessible to the content manager 322 .", + "p0125": "The content manager 322 transmits a search result to the dialog agent 324 . For example, a plurality of results may be retrieved as follows.", + "p0126": "program ABC/2013.06.04 17:00/Channel 4", + "p0127": "program ABC/2013.06.08 10:00/Channel 20", + "p0128": "program ABC/2013.06.09 20:00/Channel 35", + "p0129": "Each search result may include a title and air time of the program and channel information on the program, and further may include cast information. According to the search results, \u201cProgram ABC\u201d is broadcasted \u201cthis week\u201d on three occasions.", + "p0130": "The action agent 325 transmits the search results to the dialog engine manager 323 . In this process, because there is the plurality of search results, the action agent 325 adds a command that the display apparatus 100 displays the plurality of search results to be selectable.", + "p0131": "The dialog engine manager 323 transmits the search results and the command to the display apparatus 100 .", + "p0132": "As shown in FIG. 7 , the display apparatus 100 displays the plurality of search results to be selectable by the user according to the instruction received from the dialog engine manager 323 and directs the user to select one of the displayed search results via a system speech of the display apparatus 100 .", + "p0133": "Accordingly, the user inputs an instruction to select a first search result in a non-speech method, such as via manipulation of a button on a remote controller or a gesture. Such input is defined as a second event.", + "p0134": "The display apparatus 100 generates a virtual voice command including content of the second event corresponding to the second event. Here, the display apparatus 100 converts the second event into a form of a voice command, similar to the first event, so that the interactive server 300 provided to recognize an event in a voice command form by a speech recognizes the second event. The second event may include, for example, content \u201cNumber 1 .\u201d", + "p0135": "The display apparatus 100 transmits the second event to the dialog engine manger 323 .", + "p0136": "The dialog engine manager 323 transmits the second event to the dialog agent 324 .", + "p0137": "As the second event includes only the content \u201cNumber 1 ,\u201d the dialog agent 324 has difficulty in processing the second event only with the content of the second event. To this end, the dialog agent 324 imports processing status information on the first event from a record of processing previous events and generates a dialog frame as follows based on the imported information.", + "p0138": "Objective: To inquire about the cast of the program", + "p0139": "Order: 1", + "p0140": "Although an objective category and an attribute value thereof of the dialog frame are not included in the content of the second event, the dialog agent 324 may generate the dialog frame with the aforementioned details by referring to the first event associated with the second event.", + "p0141": "The dialog agent 324 transmits the generated dialog frame to the dialog engine manager 323 , and the dialog engine manager 323 transmits the dialog frame to the action agent 325 .", + "p0142": "The action agent 325 selects the first search result among the three search results retrieved with the first event based on an attribute value of an order condition category of the dialog frame. The action agent 325 transmits a selection result \u201cProgram ABC/2013.06.04 17:00/Channel 4\u201d to the dialog engine manager 323 .", + "p0143": "The dialog engine manager 323 transmits the selection result to the display apparatus 100 . Here, the dialog engine manager 323 may additionally transmit a system speech command to the display apparatus 100 , the system speech command being made by the dialog agent 324 with reference to the selection result.", + "p0144": "The display apparatus 100 outputs the cast of \u201cProgram ABC\u201d broadcasted on channel 4 at 2013.06.04 17:00 via a system speech based on the transmitted command.", + "p0145": "As described above, in the system including the display apparatus 100 and the interactive server 300 according to the present exemplary embodiment, when there is a plurality of results satisfying an objective and conditions input via a user's speech, even though the user uses a non-speech input method, such as a remote controller or gesture, along with a speech input method, a user-desired result is provided, thereby improving user convenience.", + "p0146": "Further, although the foregoing embodiment shows that the interactive server 300 stores a record of events respectively by a speech input and a non-speech input by the user of the display apparatus 100 , one or more exemplary embodiments are not limited thereto. For example, the display apparatus 100 may store a record of events, which will be described with reference to FIGS. 8 and 9 .", + "p0147": "FIGS. 8 and 9 are flowcharts illustrating a control method of a display apparatus 100 according to an exemplary embodiment.", + "p0148": "As shown in FIG. 8 , the display apparatus 100 may receive a speech of the user in operation S 300 . The display apparatus 100 transmits a voice command corresponding to the speech of the user to the interactive server 300 in operation S 310 .", + "p0149": "The display apparatus 100 determines whether there is a plurality of results corresponding to the voice command received from the interactive server 300 in operation S 320 .", + "p0150": "When there is a single result corresponding to the voice command, the display apparatus 100 displays the result in operation S 330 .", + "p0151": "However, when there is a plurality of results corresponding to the voice command, the display apparatus 100 displays the results to be selectable by the user in operation S 340 .", + "p0152": "The display apparatus 100 receives an input of a selection by the user among the results in operation S 350 . During this process, the display apparatus 100 may store a record of generating and processing events of the voice command by the speech.", + "p0153": "As shown in FIG. 9 , the display apparatus 100 determines whether the input of the selection is by a speech method or a non-speech method in operation S 360 .", + "p0154": "In this process, the display apparatus 100 determines whether the input of the selection received in operation S 350 is relevant to the speech in operation S 300 by referring to a pre-stored record of previous events. When the input of the selection is relevant to the speech, the display apparatus 100 performs operation S 360 .", + "p0155": "When the input of the selection is conducted by the speech method, the display apparatus 100 transmits a voice command corresponding to the input of the selection to the interactive server 300 in operation S 370 and moves to operation S 400 .", + "p0156": "When the input of the selection is conducted by the non-speech method, the display apparatus 100 generates a virtual voice command corresponding to the input of the selection in the non-speech method in operation S 380 . The display apparatus 100 transmits the virtual voice command to the interactive server 300 in operation S 390 .", + "p0157": "The display apparatus 100 receives a result corresponding to the input of the selection from the interactive server 300 and displays the result in operation S 400 .", + "p0158": "Further, the foregoing embodiment shows that the interactive server 300 analyzes and retrieves a result corresponding to an event that occurs in the display apparatus 100 . However, the idea of one of more exemplary embodiments are not limited thereto, and the display apparatus 100 may perform at least one of the functions of the STT server 200 and the interactive server 300 in the exemplary embodiment.", + "p0159": "FIG. 10 is a block diagram illustrating a configuration of a processor 620 of a display apparatus 600 according to an exemplary embodiment.", + "p0160": "As shown in FIG. 10 , the display apparatus 600 according to the present embodiment includes a communication interface 610 and the processor 620 . Other components of the display apparatus 600 are substantially the same as those mentioned in the exemplary embodiment, and thus descriptions thereof will be omitted herein.", + "p0161": "The processor 620 may include a voice processing engine 621 to process a speech input to the display apparatus 600 , a dialog engine 622 to analyze a voice command corresponding to the speech and a content manager 623 to perform retrieval based on a search keyword transmitted from the dialog engine 622 . Further, the dialog engine 622 may include a dialog engine manager 624 , a dialog agent 625 , an action agent 626 , and a database 627 . In the present embodiment illustrated in FIG. 10 , components directly associated with the idea of the exemplary embodiment are mentioned only and other elements are not shown and described herein.", + "p0162": "When a speech of the user is input, the voice processing engine 621 transmits an audio signal of the speech to the STT server 200 . The voice processing engine 621 receives a voice command corresponding to the speech from the STT server 200 and transmits the voice command to the dialog engine manager 624 .", + "p0163": "Operations of the dialog engine manager 624 , the dialog agent 625 , the action agent 626 and the content manager 623 are substantially the same as those described in the foregoing embodiment, and thus detailed descriptions thereof are omitted herein.", + "p0164": "Alternatively, the voice processing engine 621 of the display apparatus 600 may perform the function of the STT server 200 , that is, converting the speech into the voice command. In this case, when the speech of the user is input, the voice processing engine 621 converts the speech into the voice command and transmits the voice command to the dialog engine manager 624 .", + "p0165": "Although a few exemplary embodiments have been shown and described, it will be appreciated by those skilled in the art that changes may be made in these exemplary embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.", + "features": { + "F1": "The system comprises a server and at least one image processing apparatus.", + "F2": "The server includes a communicator configured to communicate with the image processing apparatus.", + "F3": "The server's processor is configured to control the communicator to transmit a plurality of retrieval results based on the input of speech received.", + "F4": "The image processing apparatus includes a display and a user interface with speech and non-speech input interfaces.", + "F5": "The image processing apparatus's communicator communicates with the server.", + "F6": "The image processing apparatus's processor controls the communicator to send the speech input to the server.", + "F7": "The display of the image processing apparatus displays the plurality of retrieval results.", + "F8": "The image processing apparatus's processor identifies the relation between non-speech input and speech input.", + "F9": "The image processing apparatus performs operations based on non-speech input independent of speech input when unrelated.", + "F10": "The image processing apparatus performs operations on retrieval results based on non-speech input related to speech input.", + "F11": "The server's processor retrieves data from a database based on speech input and transmits a final result.", + "F12": "The server's processor transmits a request command to select retrieval results with respect to the speech input.", + "F13": "The image processing apparatus converts non-speech input into a virtual voice command for the server.", + "F14": "The control method of a server includes receiving a speech input and transmitting retrieval results to an image processing apparatus.", + "F15": "The control method involves displaying retrieval results and processing non-speech input to determine its relation to speech input.", + "F16": "Retrieval is performed based on speech input and one final result is transmitted to the image processing apparatus.", + "F17": "The control method includes transmitting a request command for retrieval result selection based on non-speech input.", + "F18": "Non-speech input is converted to a virtual voice command for server recognition in the control method.", + "F19": "The image processing apparatus comprises elements for receiving speech input and communicating with the server.", + "F20": "The processor of the image processing apparatus performs operations based on the non-speech input's relation to speech input.", + "F21": "The image processing apparatus transmits converted non-speech input as a virtual voice command.", + "F22": "The control method of the image processing apparatus comprises receiving speech input and communicating with a server.", + "F23": "The method includes processing non-speech input and performing related operations on displayed retrieval results." + } + } + }, + { + "FAN": "66336898", + "CPC": [ + "G06F-016/00", + "G06F-016/9562", + "G06F-017/30", + "G06F-017/30/884", + "H04L-061/00", + "H04L-067/02", + "H04M-001/00", + "H04M-001/72457", + "H04M-001/725/72", + "H04M-2250/10", + "H04W-004/02", + "H04W-004/02/1", + "H04W-004/029" + ], + "Content": { + "title": "Location-aware management of lists of uniform resource locators for mobile devices", + "pa01": "Described herein are technologies for managing lists of universal resource locators for a mobile device based, at least in part, upon the determined location of the device. This Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.", + "c-en-0001": "A mobile device comprising: a location-awareness system configured to determine a location of the mobile device; a URL-list-manager configured to: select one or more websites that are associated with the determined location; generate a list of uniform resource locators to the one or more of the selected websites; and a contextualizer configured to determine contextual factors of the mobile device, wherein the contextual factors are selected from a group consisting of mode of travel of a user of the mobile device, crowd-sourced ratings of websites, personal history of website usage at the determined location or within a distance threshold of the determined location, crowd-sourced history of website usage at the determined location or within the distance threshold of the determined location, identification of type of the determined location, and identification of the type of event happening at the location, the URL-list-manager being further configured to select based, at least in part, upon the determined contextual factors.", + "c-en-0002": "The mobile device as recited by claim 1 , wherein the URL-list-manager is further configured to designate a group of web pages to be part of at least one of the selected websites.", + "c-en-0003": "The mobile device as recited by claim 1 , wherein the determined location of the mobile device is selected from a group consisting of a physical location, geo-location, and a logical location.", + "c-en-0004": "The mobile device as recited by claim 1 , wherein the location-awareness system is further configured to determine the location using, at least in part, geo-location information obtained from a global positioning system .", + "c-en-0005": "The mobile device as recited by claim 1 , wherein the location-awareness system is further configured to determine the location using, at least in part, location information obtained from one or more ambient identifiable wireless signal sources.", + "c-en-0006": "The mobile device as recited by claim 1 further comprising: a display configured to present thereon a user interface to a user of the mobile device, the user interface offering the generated list of ULRs to the one or more of the selected web sites; a user input system operatively associated with the user interface, the user-input system being configured to obtain input from a user that indicates the user's choice of one or more of the selected websites to access.", + "c-en-0007": "A method of management of lists of uniform resource locators for a mobile device, the method comprising: determining a location of the mobile device; determining contextual factors of the mobile device, wherein the contextual factors are selected from a group consisting of mode of travel of a user of the mobile device, crowd-sourced ratings of websites, personal history of website usage at the determined location or within a distance threshold of the determined location, crowd-sourced history of website usage at the determined location or within the distance threshold of the determined location, identification of type of the determined location, and identification of the type of event happening at the location selecting one or more websites that are associated with the determined location based, at least in part, upon the determined contextual factors; generating a list of URLs to the one or more of the selected websites.", + "c-en-0008": "The method as recited by claim 7 further comprising designating a group of web pages to be part of at least one of the selected websites.", + "c-en-0009": "The method as recited by claim 7 , wherein the determined location of the mobile device is selected from a group consisting of a physical location, geo-location, and a logical location.", + "c-en-0010": "The method as recited by claim 7 , wherein the determining of the location is based, at least in part, geo-location information obtained from a global positioning system .", + "c-en-0011": "The method as recited by claim 7 , wherein the determining of the location is based, at least in part, location information obtained from one or more ambient identifiable wireless signal sources.", + "c-en-0012": "The method as recited by claim 7 , wherein the selecting includes: querying a database to find a list of websites that are associated with the determined location; choosing one or more websites from the list of websites found by the query.", + "c-en-0013": "The method as recited by claim 12 further comprising accessing the database via a communications network.", + "c-en-0014": "The method as recited by claim 12 , wherein the database includes crowd-sourced information about websites.", + "c-en-0015": "The method as recited by claim 12 , wherein the database includes crowd-sourced information about websites, wherein such information is selected from a group consisting of usage at locations or within a distance threshold of the locations and user-supplied ratings.", + "c-en-0016": "One or more non-transitory computer-readable media with processor-executable instructions stored thereon which when executed by one or more processors cause performance of operations comprising: determining a location of a mobile device; determining contextual factors of the mobile device, wherein the contextual factors are selected from a group consisting of mode of travel of a user of the mobile device, crowd-sourced ratings of websites, personal history of website usage at the determined location or within a distance threshold of the determined location, crowd-sourced history of website usage at the determined location or within the distance threshold of the determined location, identification of type of the determined location, and identification of the type of event happening at the location; selecting one or more websites that are associated with the determined location and with one or more determined contextual factors; generating a list of uniform resource locators to the one or more of the selected websites.", + "c-en-0017": "The one or more non-transitory computer-readable media as recited by claim 16 , the operations further comprising designating a group of web pages to be part of at least one of the selected websites.", + "c-en-0018": "The one or more non-transitory computer-readable media as recited by claim 16 , wherein the determined location of the mobile device is selected from a group consisting of a physical location, geo-location, and a logical location.", + "c-en-0019": "A method comprising: determining a location of a mobile device; determining contextual factors of the mobile device, wherein the contextual factors are selected from a group consisting of mode of travel of a user of the mobile device, crowd-sourced ratings of websites, personal history of website usage at or the determined location or within a distance threshold of the determined location, personal history of website usage en route to the determined location, crowd-sourced history of website usage at the determined location or within the distance threshold of the determined location, identification of type of the determined location, and identification of the type of event happening at the location; tracking usage of one or more websites while at the determined location; generating an association between the determined location, determined contextual factors, and the one or more tracked websites; facilitating storage of the association in a database.", + "c-en-0020": "The method as recited by claim 19 , wherein the determining the contextual factors includes determining usage of one or more websites of the mobile device while at or the determined location or within the distance threshold of the determined location.", + "c-en-0021": "The method as recited by claim 20 , wherein the usage being determined for a particular website is selected from a group consisting of whether the particular website is used while at the determined location or within the distance threshold of the determined location, how much or how long the particular website is used while at the determined location or within the distance threshold of the determined location, whether the particular website is initiated while at the determined location or within the distance threshold of the determined location, whether the particular website is active while at the determined location or within the distance threshold of the determined location, whether the particular website is inactive while at the determined location or within the distance threshold of the determined location, whether the particular website is deactivated while at the determined location or within the distance threshold of the determined location, whether the particular website is installed while at the determined location or within the distance threshold of the determined location, whether the particular website is uninstalled while at the determined location or within the distance threshold of the determined location, and any combination thereof.", + "c-en-0022": "The method as recited by claim 19 , wherein the determined location of the mobile device is selected from a group consisting of a physical location, geo-location, and a logical location.", + "p0001": "BACKGROUND", + "p0002": "The use of mobile devices, such as smartphones, is nearly ubiquitous. Many of these mobile devices include the capability to determine their physical location. That is, the mobile device is capable of determining its location in the physical world. Conventionally location determination is typically accomplished by using Global Positioning Systems , some form of triangulation or interpolation of multiple radio signals, internet protocol geo-location, or some combination thereof.", + "p0003": "A collection of so-called location-based services are emerging that take advantage of the location-detection capability of the mobile devices that so many people are carrying with them each day. For example, LBSs include targeted advertising, social networking, locating friends , photo tagging, life logging, location-based games, fitness monitoring, and others. Location-based services may include vehicle or parcel tracking as well.", + "p0004": "With the ubiquitous nature of the mobile devices comes the frequent access to the websites on such devices via wireless Internet access. Users have grown accustomed to finding information by searching the World Wide Web at any time and any place.", + "p0005": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0006": "FIG. 1 shows example scenarios to illustrate implementations in accordance with the technologies described herein.", + "p0007": "FIG. 2 is a flow chart illustrating an example method in accordance with the technologies described herein.", + "p0008": "FIG. 3 is a state diagram illustrating an example method in accordance with the technologies described herein.", + "p0009": "FIG. 4 illustrates an example system in accordance with the technologies described herein.", + "p0010": "FIG. 5 illustrates an example computing device to implement in accordance with the technologies described herein.", + "p0011": "FIG. 6 illustrates an example device to implement in accordance with the technologies described herein.", + "p0012": "DETAILED DESCRIPTION", + "p0013": "The Detailed Description references the accompanying figures. In the figures, the left-most digit of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.", + "p0014": "DETAILED DESCRIPTION", + "p0015": "Disclosed herein are technologies for managing lists of uniform resource locators for a mobile device based, at least in part, upon the determined location of the device. Generally, a URL is the global address of documents, services, and other resources on the World Wide Web . A website is a set of related web pages containing content such as text, images, video, audio, etc. The web pages of a website are the most common documents to which a URL points. Consequently, a URL may also be called a link, a website address, or a web address. Collectively, a URL list may be called favorites or bookmarks.", + "p0016": "The described technology may include, for example, helping a user of a mobile device easily find URLs to websites that are appropriate and best for the current location. The disclosed technologies may also include automatic and dynamic generation of a list of URLs to location-relevant websites. Similarly, such technologies may include automatic caching of location-relevant websites when the present wireless connection to the Internet is not bandwidth restrictive or cost prohibitive.", + "p0017": "Often some websites are designed for use in specific locations or types of locations. Some examples include a university campus map, a regional subway application, or information related to a particular neighborhood or city location. An example of a website that is useful in specific types of locations is a baseball scoring website, which is useful while at a baseball game.", + "p0018": "Unfortunately, using conventional approaches, a user of a mobile device can find it difficult to find websites associated with or appropriate for a specific location and to cull the valuable ones from the less helpful ones. With the technology disclosed here, a user can arrive in a location and have his or her mobile device provide a list of links to one or more websites that are appropriate for the specific location.", + "p0019": "If the user arrives in New York City, for example, there is a tremendous number of available websites to assist in finding museums, restaurants, or even the subway schedule. Those available websites vary in degree of quality and location appropriateness. The technology described herein will help the user in finding which location-specific websites are available and which ones are valuable to that user.", + "p0020": "Another concern not adequately addressed by the conventional approaches is how to manage already cached location-specific applications based the appropriateness for the current location. When the user leaves a particular location where a location-specific website is appropriate, the technology described herein removes the location-specific website from the cache. If the user is leaving the location, then there is no need for the device to cache the web pages of that site for the user.", + "p0021": "The identification of websites that are appropriate for a particular location can also be used more generally to predict the websites that a user will access at any point in the day. As the user traverses the places and routes that he normally travels, the mobile device keeps track of the websites associated with each location .", + "p0022": "Each user of a mobile device has a limited knowledge and understanding of which location-specific websites are appropriate for a particular location. For example, a user who attends a minor league baseball game is likely unaware of a website that is particular to the ballpark that provides live statistics of the game. The user might not ever find the website by searching for it.", + "p0023": "Conventional approaches require a large amount of the user's time and manual input. When searching for websites, users can query for specific websites but they have to actively do so with keyword searches or knowledge of the type of website they are looking for. Furthermore, users must remember which websites are related to which location or try to manually arrange them in ways that makes this process easier.", + "p0024": "In short, the technology described herein helps a user to gain the benefits of using location-specific websites without requiring a large amount of manual searching for such websites.", + "features": { + "F1": "A mobile device comprises a location-awareness system configured to determine a location of the mobile device.", + "F2": "A URL-list-manager is configured to select one or more websites associated with the determined location and generate a list of uniform resource locators to the one or more selected websites.", + "F3": "A contextualizer is configured to determine contextual factors of the mobile device, where the factors are selected from mode of travel, crowd-sourced ratings, personal history of website usage, crowd-sourced history of website usage, identification of location type, and identification of the type of event at the location.", + "F4": "The location-awareness system is further configured to determine the location using geo-location information obtained from a global positioning system.", + "F5": "The location-awareness system is further configured to determine the location using information from one or more ambient identifiable wireless signal sources.", + "F6": "The device includes a display configured to present a user interface, offering the generated list of URLs, and a user input system obtaining input indicating the user's choice of websites to access.", + "F7": "A method of managing lists of uniform resource locators for a mobile device includes determining a location, determining contextual factors, selecting websites based on contextual factors, and generating a list of URLs to the selected websites.", + "F8": "The method further includes designating a group of web pages as part of at least one of the selected websites.", + "F9": "The determined location of the mobile device is selected from a group consisting of a physical location, geo-location, and a logical location.", + "F10": "The method includes querying a database to find websites associated with the determined location and choosing one or more websites from the list found by the query.", + "F11": "The database includes crowd-sourced information about websites, including usage at locations or within a distance threshold of the locations and user-supplied ratings.", + "F12": "One or more non-transitory computer-readable media have processor-executable instructions to perform operations such as determining location, determining contextual factors, selecting websites, and generating a list of URLs.", + "F13": "A method comprises tracking usage of websites at the determined location, generating an association between location, contextual factors, and tracked websites, and facilitating storage of the association in a database." + } + } + }, + { + "FAN": "105235513", + "CPC": [ + "G06F-016/00", + "G06F-016/212", + "G06F-016/22" + ], + "Content": { + "title": "Data model generation method and apparatus", + "pa01": "Provided in the embodiments of the present application are a data model generation method and apparatus, by means of which a data model closely related to a target service can be generated with relatively high accuracy and efficiency, such that an enterprise can perform independent research and development on the basis of the generated data model. The data model generation method comprises: acquiring a data access statement sent by a user end to a database, wherein the data access statement is used for accessing data of a target service in the database; according to the data access statement, acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table; determining a data type of the target column according to the identifier of the target table and the organization and structure of a table in the database; and determining a data model of the database according to the identifier of the target table, the identifier of the target column and the data type.", + "c-en-0001": "Data model generation method, wherein the method comprises: acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; determining a data type of the target column, according to the identifier of the target table and a schema of the database; determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "c-en-0002": "The method as claimed in claim 1, wherein the step of acquiring a data access statement sent by a user end to a database comprises: filtering multiple items of information generated on the basis of the target service, to obtain the data access statement, wherein the multiple items of information comprise the data access statement.", + "c-en-0003": "The method as claimed in claim 1 or 2, wherein the step of acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement, comprises: when the data access statement is a select statement, extracting a first string between a \"from\" keyword and a conditional keyword in the select statement, the first string being the identifier of the target table; and extracting a second string between a \"select\" keyword and the \"from\" keyword in the select statement, the second string being the identifier of the target column.", + "c-en-0004": "The method as claimed in claim 3, wherein, when the select statement is a join select statement and the at least one target table comprises multiple target tables, the method further comprises: acquiring a relationship of the multiple target tables on the target service, according to the join select statement; and the step of determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type, comprises: determining the data model according to the identifier of the target table, the identifier of the target column, the relationship of the multiple target tables on the target service, and the data type.", + "c-en-0005": "The method as claimed in claim 4, wherein the step of acquiring a relationship of the multiple target tables on the target service, according to the join select statement, comprises: acquiring a relationship of the multiple target tables on the target service, from a third string following the conditional keyword in the join select statement.", + "c-en-0006": "The method as claimed in claim 1 or 2, wherein the step of acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement, comprises: when the data access statement is an insert statement, extracting a fourth string between an \"insert\" keyword and a \"value\" keyword in the insert statement, the fourth string being the identifier of the target table; and extracting a fifth string in brackets following the \"value\" keyword in the insert statement, the fifth string being the identifier of the target column.", + "c-en-0007": "The method as claimed in any one of claims 1 - 6, wherein the method further comprises: subjecting the identifier of the target table and the identifier of the target column to aggregation; and the step of determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type, comprises: determining the data model on the basis of the identifier of the target table and the identifier of the target column after aggregation thereof.", + "c-en-0008": "A data model generation apparatus , comprising: a memory for storing a program; a processor for executing the program stored in the memory, wherein, when the program stored in the memory is executed, the processor is used for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; the processor is further used for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; the processor is further used for determining a data type of the target column, according to the identifier of the target table and a schema of the database; the processor is further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "c-en-0009": "A computer readable storage medium, wherein the computer readable medium stores program code for execution by a device, the program code comprising instructions for performing steps in the data model generation method as claimed in any one of claims 1 - 7.", + "c-en-0010": "A data model generation apparatus , comprising: a preprocessor , for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; an analyser , for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; a model generator , for determining a data type of the target column, according to the identifier of the target table and a schema of the database; the model generator being further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "c-en-0011": "A data model generation apparatus , comprising: an acquisition unit , for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; the acquisition unit being further used for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; a determining unit , for determining a data type of the target column, according to the identifier of the target table and a schema of the database; the determining unit being further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0001": "Technical field", + "p0002": "The present application relates to the technical field of databases, in particular to a data model generation method and apparatus.", + "p0003": "Background art", + "p0004": "At present, large and medium-sized enterprises may have large numbers of application systems, with relatively complex distribution and structure of tables of databases in the application systems. Since these application systems are generally implemented or developed by third-party companies, enterprises do not know which tables and which columns in base-layer databases are used by service modules on these application systems, and this makes future autonomous research and development by enterprises very difficult.", + "p0005": "Summary of the invention", + "p0006": "The present application provides a data model generation method and apparatus, capable of generating a data model closely related to a target service with high accuracy and efficiency, enabling an enterprise to perform autonomous research and development on the basis of the data model generated.", + "p0007": "In a first aspect, a data model generation method is provided, comprising: acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; determining a data type of the target column, according to the identifier of the target table and a schema of the database; determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0008": "When a user operates an application system , a data access statement related to the user's operation will be sent to a database from the user end. Thus, the data access statement acquired in the technical solution described above is closely related to the target service accessed by the user, and the data model generated on the basis of the data access statement is likewise closely related to the target service accessed by the user, so the problem of generating a large number of invalid data models can be effectively avoided. Furthermore, the above technical solution does not rely on manual experience, so the efficiency and accuracy with which the data model is generated are effectively improved.", + "p0009": "In some possible implementations, the step of acquiring a data access statement sent by a user end to a database comprises: filtering multiple items of information generated on the basis of the target service, to obtain the data access statement, wherein the multiple items of information comprise the data access statement.", + "p0010": "Multiple items of information can be generated on the basis of the target service, the multiple items of information comprising information other than the data access statement. The technical solution described above filters the multiple items of information, to filter out information other than the data access statement in the multiple items of information, such that all of the remaining information is the data access statement closely related to the target service, so the problem of generating a large number of invalid data models can be further avoided.", + "p0011": "In some possible implementations, the step of acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement, comprises: when the data access statement is a select statement, extracting a first string between a \"from\" keyword and a conditional keyword in the select statement, the first string being the identifier of the target table; and extracting a second string between a \"select\" keyword and the \"from\" keyword in the select statement, the second string being the identifier of the target column.", + "p0012": "In some possible implementations, when the select statement is a join select statement and the at least one target table comprises multiple target tables, the method further comprises: acquiring a relationship of the multiple target tables on the target service, according to the join select statement; and the step of determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type, comprises: determining the data model according to the identifier of the target table, the identifier of the target column, the relationship of the multiple target tables on the target service, and the data type.", + "p0013": "The technical solution described above utilizes structural information in the data access statement to determine a potential relationship between tables by analysis, and generates a data model according to this relationship. The fact that the technical solution above does not rely on a reference foreign key in a schema of the database, avoids the disadvantage of inaccuracy of the generated data model or even inability to generate a data model relationship due to the lack of a reference foreign key in the schema of the database, such that the accuracy of the generated data model is further improved.", + "p0014": "In some possible implementations, the step of acquiring a relationship of the multiple target tables on the target service, according to the join select statement, comprises: acquiring a relationship of the multiple target tables on the target service, from a third string following the conditional keyword in the join select statement.", + "p0015": "In some possible implementations, the step of acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement, comprises: when the data access statement is an insert statement, extracting a fourth string between an \"insert\" keyword and a \"value\" keyword in the insert statement, the fourth string being the identifier of the target table; and extracting a fifth string in brackets following the \"value\" keyword in the insert statement, the fifth string being the identifier of the target column.", + "p0016": "In some possible implementations, the method further comprises: subjecting the identifier of the target table and the identifier of the target column to aggregation; and the step of determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type, comprises: determining the data model on the basis of the identifier of the target table and the identifier of the target column after aggregation thereof.", + "p0017": "In the technical solution described above, before generating the data model, the identifier of the target table and the identifier of the target column are aggregated; compared with the identifier of the target table and the identifier of the target column being in a discrete state, this facilitates subsequent operations, helping to improve the efficiency of data model generation.", + "p0018": "In a second aspect, a data model generation apparatus is provided, comprising: a memory for storing a program; a processor for executing the program stored in the memory, wherein, when the program stored in the memory is executed, the processor is used to perform the method in the first aspect above or the implementations thereof.", + "p0019": "In a third aspect, a computer readable storage medium is provided, storing program code for execution by a device, the program code comprising instructions for performing steps in the method in the first aspect above or the implementations thereof.", + "p0020": "In a fourth aspect, a data model generation apparatus is provided, comprising: a preprocessor, for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; an analyser, for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; a model generator, for determining a data type of the target column, according to the identifier of the target table and a schema of the database; the model generator being further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0021": "In a fifth aspect, a data model generation apparatus is provided, comprising units for performing the method in the first aspect above or the implementations thereof.", + "p0022": "Brief description of the drawings", + "p0023": "Fig. 1 is a schematic drawing of a system architecture in an embodiment of the present application. Fig. 2 is a schematic diagram of a data model generation method in an embodiment of the present application. Fig. 3 is a flow chart of a data model generation method in an embodiment of the present application. Fig. 4 is a flow chart of another data model generation method in an embodiment of the present application. Fig. 5 is a schematic block diagram of a data model generation apparatus in an embodiment of the present application. Fig. 6 is a schematic block diagram of another data model generation apparatus in an embodiment of the present application. Fig. 7 is a schematic block diagram of another data model generation apparatus in an embodiment of the present application.", + "p0024": "Key to the drawings:", + "p0025": "110 - user end; 120 - database; 130 - third-party device; 200 - data model generation method in embodiments of the present application; 210 - acquiring a data access statement sent by a user end to a database; 220 - acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in each target table, according to the data access statement; 230 - determining a data type of the target column, according to the identifier of the target table and a schema of the database; 240 - determining a data model, according to the identifier of the target table, the identifier of the target column and the data type; Pr - preprocessor; An - analyser; Bu - model generator; Re - analysing multiple rows of information row-by-row; Fi - filtering; Di - sending SQL statement to corresponding analyser; Tn - acquiring table name of target table; Cn - acquiring column name of target column; Dt - acquiring data type of target column; Ge - generating data model; Er - acquiring relationship; As - aggregation; 500 - data model generation apparatus; 510 - preprocessor; 520 - analyser; 530 - model generator; 600 - data model generation apparatus; 610 - acquisition unit; 620 - determining unit; 700 - data model generation apparatus; 701 - memory; 702 - processor; 703 - communication interface; 704 - bus.", + "p0026": "Detailed description of the invention", + "p0027": "Technical solutions in embodiments of the present application are described below with reference to the drawings. It should be understood that the specific examples herein are merely intended to help those skilled in the art to better understand embodiments of the present application, without limiting the scope of embodiments of the present application.", + "p0028": "It should be understood that in various embodiments of the present application, the sizes of sequence numbers of each process do not signify the order of execution, and the order of execution of each process should be determined by the function and internal logic thereof, and should not constitute any limitation of the process of implementing embodiments of the present application.", + "p0029": "It should also be understood that various embodiments described herein may be implemented alone or in combination, and embodiments of the present application impose no limitations in this regard.", + "p0030": "Unless otherwise stated, all technical and scientific terms used in embodiments of the present application have the same meanings as those commonly understood by those skilled in the technical field of the present application. Terms used in the present application are merely intended to describe specific embodiments, not to limit the scope of the present application.", + "p0031": "Generally, large and medium-sized enterprises may have large numbers of application systems, with relatively complex distribution and structure of tables of databases in the application systems. Since these application systems are generally implemented or developed by third-party companies, even though the third-party companies will provide the enterprises with design files relating to the application systems, the problem will sometimes arise that the content of the design files is not comprehensive or that the instructions in the design files do not correspond exactly to the actual application systems. Thus, enterprises do not know which tables and which columns in base-layer databases are used by service modules on these application systems, and this makes future autonomous research and development by enterprises very difficult. For example, based on data of an existing application system, enterprises must subject the application system to information upgrade/modification or service expansion.", + "p0032": "At present, two methods are widely used in the industry to solve the abovementioned problems, the first being a method based on manual analysis. Specifically, the method mainly relies on manual work to sort through data and columns on interactive table forms of various application systems when processing various services, and find data and columns of tables in a database that match said data and columns. This information is then gathered together and tidied up, to finally perform analysis to determine tables of databases that application systems might call, and corresponding columns. For example, the application system is a financial system, and a user needs to query the sales volume of each quarter in one year; after the user's query, data outputted by the financial system is \"A\", \"B\", \"C\" and \"D\", and the user can then reverse-search for \"A\", \"B\", \"C\" and \"D\" in a database, or determine a few items of data having features amongst the four items of data \"A\", \"B\", \"C\" and \"D\", such as \"C\" and \"D\", then search for \"C\" and \"D\" in the database, and finally perform analysis to determine a database table that the financial system might call, and a corresponding column.", + "p0033": "This method primarily relies on manual experience, so its efficiency and accuracy are relatively low, especially in cases where a database comprises a large number of tables. In addition, since the computation logic after submission of the application system table form might involve processing and aggregation querying of the structure of intermediate tables or a large number of tables, this querying and processing cannot be visually seen directly on a user interface, so the method of manual analysis might overlook many potential data models, with the result that the data model obtained is not very accurate.", + "p0034": "The second method is an analysis method based on the organization and structure of tables in a database . Specifically, by calling associated interfaces of databases corresponding to application systems, the method acquires the relationships and structures of tables of all databases in the application systems at present, and finally reverse-generates a data model.", + "p0035": "Since this method is unable to dynamically perceive which tables in a database have been accessed for a particular service, it is unable to help the user to effectively determine a data model currently used by an application system by performing sorting. For example, a database contains 1000 tables, and when a user accesses data of a particular service, an application system might only use 100 of these tables, but because the method is unable to learn which 100 of the 1000 tables are used, it has to subject all 1000 tables to reverse data model generation, which takes a long time and has low efficiency. In addition, since a database might contain data not generated by human-machine interaction or data unrelated to the service currently accessed by the user, the problem may arise that the data model generated deviates from the service and is unrelated to the service currently accessed by the user.", + "p0036": "Further, relationships in data models generated by the method are highly reliant on reference foreign keys of tables of databases. If a table does not contain information of this type, a relationship of a data model cannot be generated. It has been proven through a large number of actual production environments that the larger the enterprise system, the less it contains strong reference foreign key relationships of this type. Therefore, the method is not very feasible in actual production environments.", + "p0037": "In view of this, embodiments of the present application propose a data model generation method, which is capable of generating a data model closely related to a target service with high accuracy and efficiency, enabling an enterprise to perform autonomous research and development on the basis of the data model generated.", + "p0038": "Fig. 1 is a schematic diagram of a system architecture in an embodiment of the present application.", + "p0039": "The system architecture shown in Fig. 1 comprises a user end 110, a database 120 and a third-party device 130. The user end 110 may be an interface used to provide data query service management. The user end 110 may also be called a client end or another name.", + "p0040": "The database 120 may for example be mysql, oracle, sqlserver, sqlite, etc. Information may be transmitted between the user end 110 and the database 120, for example, when the user needs to query data, e.g. query sales volume of each quarter in one year, query information may be inputted at the user end 110, and after receiving the query information, the user end 110 converts the query information to database language, such as structured query language , and sends the database language to the database 120. The database 120 then determines, by querying, the sales volume of each quarter in one year on the basis of the received database language, and sends the query result to the user end 110. The user end 110 can then output the query result to the user, and the user can thus obtain the sales volume of each quarter in one year.", + "p0041": "The third-party device 130 may be in communicative connection with the user end 110 and the database 120, so as to acquire information transmitted between the user end 110 and the database 120, and perform some operations on the basis of the information, such as generating a data model of the database 120. Specifically, the third party 130 may comprise a communication interface, to realize a communicative connection with another device . The communicative connection may be wired or wireless. The third-party device 130 may acquire the information during information transmission; or, if the user end 110 sends information to the database 120, the third-party device 130 may acquire the information from the database 120 after the information has been sent to the database 120.", + "p0042": "Demonstratively, the third-party device 130 may be a server. The server is a device providing a computing service, and the composition of the server comprises a processor, a hard disk, an internal memory, a system bus, etc.; the server has a similar architecture to a general-purpose computer, but due to the need to provide a highly reliable service, it is subject to higher requirements in terms of processing ability, stability, reliability, security, expandability, manageability, etc.", + "p0043": "It should be understood that Fig. 1 is merely a schematic drawing of one system architecture provided in embodiments of the present application, and the positional relationships among equipment, devices and modules, etc. shown in the figure do not constitute any limitation.", + "p0044": "The main process of the data model generation method in embodiments of the present application is presented below with reference to Fig. 2 .", + "p0045": "Fig. 2 shows a schematic flow chart of a data model generation method 200 in an embodiment of the present application. The method 200 may be performed by a third-party device other than a user end and a database, such as the third-party device 130 shown in Fig. 1 . As shown in Fig. 2 , the method 200 may comprise at some of the following content.", + "p0046": "Step 210: acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database.", + "p0047": "Step 220: acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in each target table, according to the data access statement.", + "p0048": "Step 230: determining a data type of the target column, according to the identifier of the target table and a schema of the database.", + "p0049": "Step 240: determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0050": "When a user operates an application system , a data access statement related to the user's operation will be sent to the database from the user end. Thus, the data access statement acquired in embodiments of the present application is closely related to the target service accessed by the user, and the data model generated on the basis of the data access statement is likewise closely related to the target service accessed by the user, so the problem of generating a large number of invalid data models can be effectively avoided. Furthermore, the above technical solution does not rely on manual experience, so the efficiency and accuracy with which the data model is generated are effectively improved.", + "p0051": "In embodiments of the present application, the third-party device may comprise a preprocessor, an analyser and a model generator. Demonstratively, step 210 may be performed by the preprocessor, step 220 may be performed by the analyser, and steps 230 and 240 may be performed by the model generator. Of course, the third-party device may also comprise other devices, such as a data type processor. In this case, step 210 may be performed by the preprocessor, step 220 may be performed by the analyser, step 230 may be performed by the data type processor, and step 240 may be performed by the model generator. To facilitate description, the case where step 210 is performed by the preprocessor, step 220 is performed by the analyser, and steps 230 and 240 are performed by the model generator will be taken as an example below to describe the method in embodiments of the present application.", + "p0052": "As an example, the preprocessor may acquire the data access statement by means of a plug-in provided by the database, such as a command line tool.", + "p0053": "As another example, the preprocessor may activate an option, and may then obtain, by listening, the data access statement sent to the database by the user end, and is thereby able to acquire the data access statement.", + "p0054": "Optionally, the preprocessor may acquire the data access statement from the database, or may acquire the data access statement during transmission of the data access statement from the user end to the database.", + "p0055": "Optionally, the data access statement may include but is not limited to an SQL statement. Embodiments of the present application are explained taking an SQL statement as an example. SQL statements may comprise data definition language and data manipulation language ; the SQL statement in embodiments of the present application is a DML statement. The DML statement may comprise an insert statement, a delete statement, an update statement and a query statement. We take into account the fact that the granularity of delete statements and update statements is too small; for example, they generally delete one column or update one column, and consequently, it might be impossible to determine a data model by analysis on the basis of a delete statement and an update statement. Therefore, the SQL statement in embodiments of the present application is a select statement or an insert statement.", + "p0056": "If the SQL statement is acquired by the preprocessor from the database, we take into account the fact that, after the database has received the SQL statement sent by the user end, the database may generate context information based on the SQL statement, e.g. generate a time stamp, a session ID or a process ID, etc. That is to say, the database may generate multiple items of information based on the target service, the multiple items of information comprising information other than multiple SQL statements. Moreover, even if it is an SQL statement, it might comprise a delete statement and an update statement.", + "p0057": "Alternatively, if the SQL statement is acquired during transmission from the user end to the database, then in addition to the SQL statement, the user end might also transmit other information to the database.", + "p0058": "Thus, step 210 may specifically comprise: the preprocessor filtering multiple items of information generated on the basis of the target service, filtering out information other than SQL statements in the multiple items of information and filtering out delete statements and update statements among SQL statements, to obtain a select statement or an insert statement.", + "p0059": "This technical solution filters the multiple items of information, to filter out information other than the data access statement in the multiple items of information, such that all of the remaining information is the data access statement closely related to the target service, so the problem of generating a large number of invalid data models can be further avoided.", + "p0060": "The preprocessor might simultaneously receive multiple rows of information; thus, optionally, as shown in Figs. 3 and 4 , the preprocessor may also subject the multiple rows of information received to row-by-row analysis, and then filter each row of information, to obtain the SQL statement.", + "p0061": "Optionally, in embodiments of the present application, the identifier of the target table may be a table name of the target table, or an ID of the target table, or a serial number of the target table among all tables in the database. Similar to the identifier of the target table, the identifier of the target column may be a column name of the target column, or an ID of the target column, or a serial number of the target column in the target table. In each case below, the identifier of the target table is the table name and the identifier of the target column is the column name, as an example, for the purpose of illustration.", + "p0062": "The analyser acquires the table name of the target table and the column name of the target column in different ways for select statements and insert statements.", + "p0063": "For a select statement, specifically, the analyser may extract a first string between the keyword \"from\" and a conditional keyword in the select statement, and the first string is the table name of the target table. For example, a select statement is \"select coll, col2, col3 from table_1 where...\"; the string between the keyword \"from\" and the conditional keyword in this select statement is table_1, so the table name extracted is table_1. As another example, the select statement is \"select coll, col2, col3 from table_1 left join table_2 on...\"; the string between the keyword \"from\" and the conditional keyword in this select statement is table_1 and table_2, so the table name extracted is table_1 and table_2.", + "p0064": "If the analyser extracts multiple first strings, the multiple first strings may be separated by a comma, i.e. a set of table names may be obtained, e.g. above.", + "p0065": "Furthermore, the analyser may extract a second string between the keyword \"select\" and the keyword \"from\" in the select statement, and the second string is the column name of the target column. If there are multiple column names, the multiple column names may likewise be separated by a comma. For example, the select statement is \"select coll, col2, col3 from table_1 where...\"; the string between the keyword \"select\" and the keyword \"from\" in this select statement is \"coll, col2, col3\", so the column name of the target column is .", + "p0066": "In the case of an insert statement, the analyser may extract a fourth string between the keywords \"insert into\" and the keyword \"values\" in the insert statement, and the fourth string is the table name of the target table. Furthermore, the analyser may extract a fifth string in brackets following the keyword \"values\" in the insert statement, and the fifth string is the column name of the target column.", + "p0067": "Furthermore, if the select statement is a join select statement, such as the statement \"select coll, col2, col3 from table_1 left join table_2 on...\" above, and the target table comprises multiple target tables, then as shown in Fig. 4 , the method 200 may further comprise: the analyser acquiring a relationship of multiple target tables on the target service according to the join select statement. In other words, the analyser may acquire by which columns the multiple target tables are associated with each other on the target service.", + "p0068": "Specifically, the analyser may acquire the relationship of multiple target tables on the target service from a third string following the conditional keyword in the join select statement. For example, the join select statement is \"select coll, col2, col3 from table_1 left join table_2 on table_1.col1 = table_2.col2; as can be seen, there is a relationship between table_1 and table_2, and table_1 and table_2 are associated with each other via the columns with column names coll and col2, i.e. the value of the column with column name coll in table_1 and the value of the column with column name col2 in table_2 refer to each other, and the value of the column with column name col2 in table_2 is derived from the value in the column with column name coll in table_1.", + "p0069": "After determining the relationship between table_1 and table_2, it is possible to string together table1, coll, table2 and col2 to form a four-element set . The four-element set can indicate that table_1 and table_2 are associated with each other via the column with column name coll in table_1 and the column with column name col2 in table_2.", + "p0070": "If the join select statement comprises multiple instances of \"join\" being used consecutively, third strings following the conditional keyword may be extracted separately, to obtain a multi-dimensional four-element array. For example:", + "p0071": ", , , ]", + "p0072": "In this case, in addition to being able to determine the data model according to the table name of the target table, the column name of the target column, and the data type, the analyser may also determine the data model according to the relationship of the multiple tables on the target service.", + "p0073": "The above technical solution utilizes structural information in the data access statement to determine a potential relationship between tables by analysis, and generates a data model according to this relationship. The fact that the technical solution above does not rely on a reference foreign key in a schema of the database, avoids the disadvantage of inaccuracy of the generated data model or even inability to generate a data model relationship due to the lack of a reference foreign key in the schema of the database, such that the accuracy of the generated data model is further improved.", + "p0074": "It should be understood that in embodiments of the present application, \"first\", \"second\", \"third\" and \"fourth\" are merely intended to distinguish between different objects, but do not limit the scope of embodiments of the present application.", + "p0075": "It should also be understood that the specific examples in embodiments of the present application are merely intended to help those skilled in the art to better understand embodiments of the present application, without limiting the scope of embodiments of the present application.", + "p0076": "It can be seen from the description above that the analyser acquires the table name of the target table and the column name of the target column in different ways for select statements and insert statements. Thus, in an embodiment, there may be two analysers; one of the analysers may acquire the table name of the target table, the column name of the target column and the relationship of multiple target tables on the target service in the select statement, and the other analyser may acquire the table name of the target table and the column name of the target column in the insert statement.", + "p0077": "For example, as shown in Figs. 3 and 4 , the analyser in Fig. 3 is the second analyser, which can acquire the table name and column name; and the analyser in Fig. 4 is the first analyser, which can acquire the table name, the column name, and the relationship between tables.", + "p0078": "In this embodiment, before step 220, the preprocessor may determine the type of the SQL statement, and then send the SQL statement to the corresponding analyser. That is, it sends the select statement to the first analyser in Fig. 4 , and sends the insert statement to the second analyser in Fig. 3 .", + "p0079": "In another embodiment, there may be three analysers; one of the analysers may acquire the table name and column name of the join select statement, another analyser may acquire the table name and column name of a statement other than the join select statement among select statements, and the remaining analyser may acquire the table name and column name of the insert statement.", + "p0080": "In another embodiment, there may be one analyser, for example the analyser in Fig. 4 . This analyser may perform three steps in total: acquiring the table name of the target table in the SQL statement, acquiring the column name of the target column, and acquiring the relationship of multiple target tables on the target service. If the SQL statement is a join select statement, the analyser may perform these three steps in succession; if the SQL statement is another statement , the analyser may not perform the step of acquiring the relationship of multiple target tables on the target service, and only perform the first two steps.", + "p0081": "To facilitate subsequent model generator processing, optionally, as shown in Fig. 4 , the method 200 may further comprise: subjecting the table name of the target table and the column name of the target column to aggregation, or subjecting the table name of the target table, the column name of the target column and the relationship of multiple target tables on the target service to aggregation.", + "p0082": "The aggregated table name and column name may be called a structured result set, or the aggregated table name, column name and relationship may be called a structured result set. The structured result set may be understood to describe an object in a non-discrete manner. For example, when describing what articles a supermarket has, the description may be that fruit includes apples and bananas, etc., daily necessities include laundry liquid and towels, etc., and vegetables include potatoes and tomatoes, etc., instead of the description that the supermarket has bananas, laundry detergent and potatoes, etc.", + "p0083": "Demonstratively, suppose that there are two target tables, with table names table_X and table_Y respectively, wherein the column names of target columns in table_X are coll, col2 and colN respectively, the column names of target columns in table_Y are coll, col2 and colN respectively, and table_X and table_Y are associated with each other via the column with column name coll in table_X and the column with column name col2 in table_Y. After aggregation of the table names of the target tables, the column names of the target columns, and the relationship of the two target tables, a JSON compound structural body as shown below may be obtained: , ],\"relations\": }", + "p0084": "In the technical solution above, before generating the data model, the identifiers of the target tables and the identifiers of the target columns are aggregated; compared with the identifiers of the target tables and the identifiers of the target columns being in a discrete state, this facilitates subsequent operations, helping to improve the efficiency of data model generation.", + "p0085": "After the analyser has determined the column names of the target columns, the model generator still needs to determine the data types of the target columns. For example, whether the data type of a target column is an integer type or a string type, etc. Specifically, based on the table name of a target table, the model generator can extract from the schema of the database a data type of a target column in said target table, and can thereby obtain a list of three-element sets, e.g. , , ]. Here, \"datatype\" represents the data type.", + "p0086": "After obtaining the data types the target columns, the model generator can determine a data model of the database, based on the table names of the target tables, the column names of the target columns and the data types, or based on the table names of the target tables, the column names of the target columns, the relationship of multiple target tables on the target service, and the data types.", + "p0087": "Optionally, the table name of the target table may act as an entity name of the data model, the column name of the target column may act as an attribute name of the corresponding entity, and the data type of the target column may act as a data type of the corresponding entity attribute.", + "p0088": "Suppose that there are four target tables called by a target service, the table names being respectively Table_X, Table_Y, Table_Z and Table_N, wherein the column name of the target column in Table_X is Col_X, and the data type of the target column is Datatype_X; the column name of the target column in Table_Y is Col_Y, and the data type of the target column is Datatype_Y; the column name of the target column in Table_Z is Col_Z, and the data type of the target column is Datatype_Z; the column name of the target column in Table_N is Col_N, and the data type of the target column is Datatype_N. The generated data model may be as shown in Table 1.", + "p0089": "Table 1 Entity Name Entity Attribute Name Entity Attribute Data Type Table_X Col_X Datatype_X Table_Y Col_Y Datatype_Y Table_Z Col_Z Datatype_Z Table_N Col_N Datatype_N", + "p0090": "Furthermore, if multiple target tables have a relationship on the target service, the table names of multiple mutually associated target tables may be used as entity names of multiple mutually associated entities, and the column names of multiple mutually associated target columns may be used as association attributes of multiple mutually associated entities.", + "p0091": "An explanation is now given, taking Table 1 as an example. Suppose that Table_X and Table_Y are mutually associated via the column with column name Col_X in Table_X and the column with column name Col_Y in Table_Y, and that Table_Z and TableN are mutually associated via the column with column name Col_Z in Table_Z and the column with column name Col_Z in Table_N; then the relationships between multiple entities obtained may be as shown in Table 2.", + "p0092": "Table 2 From_Entity From_Attribute_Name To_Entity To_Attribute_Name Table_X Col_X Table_Y Col_Y Table_N-1 Col_N-1 Table_N Col_N", + "p0093": "Here, the values of from_tab and to_tab are the entity names of two mutually associated entities. from_col and to_col are the association attributes of two mutually associated entities.", + "p0094": "A data model finally generated according to an insert statement and a non-join select statement among select statements is as shown in Table 1; a data model finally generated according to a join select statement is as shown in Table 1 and Table 2.", + "p0095": "It should be explained that if entities with the same name are present, the analyser may also subject the entities with the same name to attribute expansion.", + "p0096": "Optionally, the method 200 may further comprise: making the data model persistent, to store the data model in a base-layer database.", + "p0097": "Method embodiments among embodiments of the present application have been described in detail above; apparatus embodiments among embodiments of the present application are described below. The apparatus embodiments correspond to the method embodiments; therefore, for parts not described in detail, reference may be made to the method embodiments above. The apparatus is able to realize any possible manner in the method above.", + "p0098": "Fig. 5 shows a schematic block diagram of a data model generation apparatus 500 in an embodiment of the present application. The data model generation apparatus 500 can perform the data model generation method 200 in embodiments of the present application described above, and the data model generation apparatus 500 may be the third-party device in the method above.", + "p0099": "As shown in Fig. 5 , the data model generation apparatus 500 may comprise:", + "p0100": "a preprocessor 510, for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database; an analyser 520, for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement; a model generator 530, for determining a data type of the target column, according to the identifier of the target table and a schema of the database.", + "p0101": "The model generator 530 is further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0102": "Optionally, in an embodiment of the present application, the preprocessor 510 is specifically used for: filtering multiple items of information generated on the basis of the target service, to obtain the data access statement, wherein the multiple items of information comprise the data access statement.", + "p0103": "Optionally, in an embodiment of the present application, the analyser 520 is specifically used for: when the data access statement is a select statement, extracting a first string between a \"from\" keyword and a conditional keyword in the select statement, the first string being the identifier of the target table; and extracting a second string between a \"select\" keyword and the \"from\" keyword in the select access statement, the second string being the identifier of the target column.", + "p0104": "Optionally, in an embodiment of the present application, when the select statement is a join select statement and the at least one target table comprises multiple target tables, the analyser 520 is specifically used for: acquiring a relationship of the multiple target tables on the target service, according to the join select statement; and determining the data model according to the identifier of the target table, the identifier of the target column, the relationship of the multiple target tables on the target service, and the data type.", + "p0105": "Optionally, in an embodiment of the present application, the analyser 520 is specifically used for: acquiring a relationship of the multiple target tables on the target service, from a third string following the conditional keyword in the join select statement.", + "p0106": "Optionally, in an embodiment of the present application, the analyser 520 is specifically used for: when the data access statement is an insert statement, extracting a fourth string between an \"insert\" keyword and a \"value\" keyword in the insert statement, the fourth string being the identifier of the target table; and extracting a fifth string in brackets following the \"value\" keyword in the insert statement, the fifth string being the identifier of the target column.", + "p0107": "Optionally, in an embodiment of the present application, the analyser 520 is further used for: subjecting the identifier of the target table and the identifier of the target column to aggregation; and the model generator 530 is specifically used for: determining the data model on the basis of the identifier of the target table and the identifier of the target column after aggregation thereof.", + "p0108": "It should be understood that the preprocessor 510, the analyser 520 and the model generator 530 may respectively realize the corresponding operations of the preprocessor, the analyser and the model generator in the method 200, which for brevity are not described again here.", + "p0109": "Fig. 6 shows a schematic block diagram of a data model generation apparatus 600 in another embodiment of the present application. The data model generation apparatus 600 can perform the data model generation method 200 in embodiments of the present application described above, and the data model generation apparatus 600 may be the third-party device in the method above.", + "p0110": "As shown in Fig. 6 , the data model generation apparatus 600 may comprise: an acquisition unit 610, for acquiring a data access statement sent by a user end to a database, the data access statement being used to access data of a target service in the database.", + "p0111": "The acquisition unit 610 is further used for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement.", + "p0112": "A determining unit 620 is used for determining a data type of the target column, according to the identifier of the target table and a schema of the database.", + "p0113": "The determining unit 620 is further used for determining a data model of the database, according to the identifier of the target table, the identifier of the target column and the data type.", + "p0114": "Optionally, in an embodiment of the present application, the acquisition unit 610 is specifically used for: filtering multiple items of information generated on the basis of the target service, to obtain the data access statement, wherein the multiple items of information comprise the data access statement.", + "p0115": "Optionally, in an embodiment of the present application, the acquisition unit 610 is specifically used for: when the data access statement is a select statement, extracting a first string between a \"from\" keyword and a conditional keyword in the select statement, the first string being the identifier of the target table; and extracting a second string between a \"select\" keyword and the \"from\" keyword in the select statement, the second string being the identifier of the target column.", + "p0116": "Optionally, in an embodiment of the present application, when the select statement is a join select statement and the at least one target table comprises multiple target tables, the acquisition unit 610 is specifically used for: acquiring a relationship of the multiple target tables on the target service, according to the join select statement; and determining the data model according to the identifier of the target table, the identifier of the target column, the relationship of the multiple target tables on the target service, and the data type.", + "p0117": "Optionally, in an embodiment of the present application, the acquisition unit 610 is specifically used for: acquiring a relationship of the multiple target tables on the target service, from a third string following the conditional keyword in the join select statement.", + "p0118": "Optionally, in an embodiment of the present application, the acquisition unit 610 is specifically used for: when the data access statement is an insert statement, extracting a fourth string between an \"insert\" keyword and a \"value\" keyword in the insert statement, the fourth string being the identifier of the target table; and extracting a fifth string in query brackets of the \"value\" keyword in the insert statement, the fifth string being the identifier of the target column.", + "p0119": "Optionally, in an embodiment of the present application, the data model generation apparatus 600 further comprises an aggregation unit, for subjecting the identifier of the target table and the identifier of the target column to aggregation; and the determining unit 620 is specifically used for: determining the data model on the basis of the identifier of the target table and the identifier of the target column after aggregation thereof.", + "p0120": "Fig. 7 is a schematic drawing of the hardware structure of a data model generation apparatus 700 in an embodiment of the present application. The data model generation apparatus 700 shown in Fig. 7 may be a third-party device, and the data model generation apparatus 700 comprises a memory 701, a processor 702, a communication interface 703 and a bus 704. The memory 701, processor 702 and communication interface 703 are communicatively connected to one another via the bus 704.", + "p0121": "The memory 701 may be read-only memory , a static storage devices and random access memory . The memory 701 may store a program, and when the program stored in the memory 701 is executed by the processor 702, the processor 702 and the communication interface 703 are used to perform various steps of the data model generation method in embodiments of the present application.", + "p0122": "The processor 702 may be a general-purpose CPU, a microprocessor, an application specific integrated circuit , a graphics processing unit , or one or more integrated circuit, used to perform a relevant program, so as to realize functions required to be performed by units in the apparatus in embodiments of the present application, or perform the data model generation method in embodiments of the present application.", + "p0123": "The processor 702 may also be an integrated circuit chip, with the ability to process signals. In the process of implementation, various steps of the data model generation method in embodiments of the present application may be completed by means of instructions in the form of software or integrated logic circuitry of hardware in the processor 702.", + "p0124": "The processor 702 may also be a general-purpose processor, a digital signal processor , an ASIC, a field programmable gate array , or another programmable logic device, discrete gate or transistor logic device/discrete hardware component. It is able to realize or execute each method, step and logic block diagram disclosed in embodiments of the present application. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. Steps of the method disclosed in conjunction with embodiments of the present application may be directly embodied as hardware for the processor to execute to completion, or be executed to completion using a combination of hardware and software modules in the processor. The software modules may be located in a storage medium that is mature in the art, such as RAM, internal memory, ROM, programmable ROM or electrically erasable programmable memory/registers. The storage medium is located in the memory 701; the processor 702 reads information in the memory 701, and in conjunction with hardware thereof, completes functions required to be performed by units comprised in the apparatus in embodiments of the present application, or performs the data model generation method in embodiments of the present application.", + "p0125": "The communication interface 703 uses a transceiving apparatus such as, but not limited to, a transceiver, to realize communication between the data model generation apparatus 700 and another device or communication network.", + "p0126": "The bus 704 may comprise a pathway for transmitting information among the various components of the data model generation apparatus 700.", + "p0127": "It should be noted that although only the memory, processor and communication interface are shown in the data model generation apparatus 700 described above, those skilled in the art will understand that in the process of specific implementation, the data model generation apparatus 700 may also comprise other devices necessary for normal operation. Moreover, those skilled in the art will understand that the data model generation apparatus 700 may also comprise hardware devices for realizing other additional functions, depending on actual needs. Furthermore, those skilled in the art will understand that the data model generation apparatus 700 may also only comprise devices that are necessary for realizing embodiments of the present application, and need not comprise all of the devices shown in Fig. 7 .", + "p0128": "Embodiments of the present application further provide a computer readable storage medium storing program code for execution by a device, the program code comprising instructions for executing steps in the data model generation method described above.", + "p0129": "Embodiments of the present application further provide a computer program product, comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the data model generation method described above.", + "p0130": "The computer readable storage medium may be a transient computer readable storage medium or a non-transient computer readable storage medium.", + "p0131": "Those skilled in the art will clearly understand that for convenience and brevity of description, the specific operating process of the apparatus described above is not described here superfluously, as reference may be made to the corresponding process in the method embodiments above.", + "p0132": "In several embodiments provided in the present application, it should be understood that the apparatus and method disclosed may be realized in other ways. For example, the apparatus embodiments described above are merely schematic; for example, the division of the units is merely a division in terms of logic functions, and division may be accomplished in other ways in actual implementation, e.g. multiple units or components may be combined or may be integrated in another system, or certain features may be omitted or not executed. Furthermore, couplings or direct couplings or communicative connections between parts that are shown or discussed may be indirect couplings or communication connections via certain interfaces, apparatuses or units, and may be electrical, mechanical or in another form. Words used in the present application are merely used for describing embodiments, not for limiting the claims. As used in the description of the claims and embodiments, unless clearly indicated otherwise in the context, the singular forms \"a\" and \"the\" are intended to likewise include the plural forms. Similarly, as used in the present application, the term \"and/or\" means any and all possible combinations including one or more than one of the associated items listed. Furthermore, when used in the present application, the term \"comprises\" refers to the existence of the stated feature, entity, step, operation, element and/or component, without ruling out the existence or addition of one or more other feature, entity, step, operation, element, component and/or subgroup of these.", + "p0133": "All aspects, embodiments, implementations or features in the embodiments described may be used alone or in any combination. All aspects in the embodiments described may be realized by software, hardware or a combination of software and hardware. The embodiments described may also be embodied by a computer readable medium storing computer readable code, the computer readable code comprising instructions executable by at least one computing apparatus. The computer readable medium may be associated with any data storage apparatus capable of storing data readable by a computer system. Examples of computer readable media may include ROM, RAM, compact disc read-only memory , hard disk drives , digital video discs , magnetic tape and optical data storage apparatuses, etc. The computer readable media may also be distributed in a computer system connected via a network, such that the computer readable code may be stored and executed in a distributed manner.", + "p0134": "The technical descriptions allow reference to the drawings, which form part of the present application, and implementations according to the described embodiments are shown in the drawings by description. Although these embodiments are described in sufficient detail to enable a person skilled in the art to implement them, they are non-limiting; thus, other embodiments may be used, and changes may be made without departing from the scope of the described embodiments. For example, the order of operations described in the flow chart is non-limiting, so the order of two or more operations illustrated in the flow chart and described in accordance with the flow chart may be changed according to embodiments. As another example, in some embodiments, one or more operation illustrated in the flow chart and described in accordance with the flow chart is optional, or may be deleted. Furthermore, some steps or functions may be added to the disclosed embodiments, or the order of two or more steps is switched. All of these changes are considered to be included in the disclosed embodiments and claims.", + "p0135": "The above are merely specific ways of implementing embodiments of the present application, but the scope of protection of embodiments of the present application is not limited to this. All changes or substitutions which might readily occur to any person skilled in the art within the technical scope of disclosure in embodiments of the present application shall be included in the scope of protection of embodiments of the present application. Therefore, the scope of protection of embodiments of the present application shall be the scope of protection of the claims.", + "features": { + "F1": "A data model generation method includes acquiring a data access statement sent by a user end to a database, where the data access statement is used to access data of a target service in the database.", + "F2": "The method involves acquiring an identifier of each of at least one target table in the database called by the target service, and an identifier of a target column in the target table, according to the data access statement.", + "F3": "The method determines a data type of the target column, according to the identifier of the target table and a schema of the database.", + "F4": "The method determines a data model of the database, based on the identifier of the target table, the identifier of the target column, and the data type.", + "F5": "The data access statement can be acquired by filtering multiple items of information generated on the basis of the target service, where the multiple items of information include the data access statement.", + "F6": "When the data access statement is a select statement, the method extracts a first string between a 'from' keyword and a conditional keyword in the select statement as the identifier of the target table, and a second string between a 'select' keyword and the 'from' keyword as the identifier of the target column.", + "F7": "For a join select statement with multiple target tables, the method involves acquiring a relationship between the multiple target tables on the target service according to the join select statement.", + "F8": "The determination of the data model in a join select statement considers the identifier of the target table, the identifier of the target column, the relationship of the multiple target tables on the target service, and the data type.", + "F9": "In an insert statement, the method extracts a fourth string between an 'insert' keyword and a 'value' keyword as the identifier of the target table, and a fifth string in brackets following the 'value' keyword as the identifier of the target column.", + "F10": "The method includes subjecting the identifier of the target table and the identifier of the target column to aggregation.", + "F11": "The method infers and determines the data model based on the identifier of the target table and the identifier of the target column after their aggregation.", + "F12": "A data model generation apparatus comprises a memory for storing a program and a processor for executing the program to acquire a data access statement sent by a user end to a database.", + "F13": "The processor in the apparatus is used for acquiring an identifier of each of at least one target table in the database which is called by the target service, and an identifier of a target column in the target table, according to the data access statement.", + "F14": "The processor also determines a data type of the target column, according to the identifier of the target table and a schema of the database.", + "F15": "The processor determines a data model of the database, based on the identifier of the target table, the identifier of the target column, and the data type.", + "F16": "A computer readable storage medium stores program code for executing the steps in the data model generation method.", + "F17": "A data model generation apparatus includes a preprocessor for acquiring a data access statement, an analyser for acquiring table and column identifiers, and a model generator for determining a data type and a data model of the database.", + "F18": "A data model generation apparatus comprises an acquisition unit for acquiring a data access statement and identifiers of target table and column, and a determining unit for determining a data type and a data model of the database." + } + } + }, + { + "FAN": "79740635", + "CPC": [ + "G06F-016/00", + "G06F-016/2379", + "G06F-016/24578", + "G06F-016/248", + "G06F-016/907" + ], + "Content": { + "title": "Matching device, matching method, and program", + "pa01": "A matching apparatus includes: a sensor-side metadata acquisition unit that acquires sensor-side metadata; an application-side metadata acquisition unit that acquires application-side metadata; a matching unit that performs matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each other, and extracts a sensor candidate; and a dataflow control command instruction unit that transmits a dataflow control command that includes information that specifies the sensor extracted by the matching unit and an application.", + "c-en-0001": "A matching apparatus comprising a processor configured with a program to perform operations comprising: operation as a sensor-side metadata acquisition unit that acquires sensor-side metadata comprising information regarding a sensor that outputs sensing data; operation as an application-side metadata acquisition unit that acquires application-side metadata comprising information regarding an application that provides a service with use of the sensing data; operation as a matching unit that performs matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each other, and extracts a candidate sensor that provides sensing data that matches needs of the application, wherein operation as the matching unit comprises: operation as a relevancy calculation unit that calculates at least either one of a first relevancy between the term in the sensor-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the second related term; and operation as a sensor candidate extraction unit that, with the use of at least either one of the first related term and the first relevancy or the second related term and the second relevancy, performs matching between the sensor-side metadata and the application-side metadata, and extracts the candidate sensor that can provide sensing data that matches needs of the application; and operation as an instruction unit that transmits, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the sensor extracted by the matching unit and the application.", + "c-en-0002": "The matching apparatus according to claim 1 , wherein the processor is configured with the program such that operation as the sensor candidate extraction unit comprises performing matching between the sensor-side metadata and the application-side metadata with use of the first related term having a first relevancy that is in a predetermined range and the first relevancy, or with use of the second related term having a second relevancy that is in a predetermined range and the second relevancy.", + "c-en-0003": "The matching apparatus according to claim 2 , wherein the processor is configured with the program to perform operations further comprising: operation as a range setting unit that sets the predetermined range of the first relevancy to be used with respect to the first related term, or the predetermined range of the second relevancy to be used with respect to the second related term.", + "c-en-0004": "The matching apparatus according to claim 3 , wherein the processor is configured with the program such that, in a case where a contract is established with an application-side user through matching, operation as the range setting unit comprises setting, as a range of relevancy of a related term to be used in subsequent matching processing, a range of relevancy of a related term that was used when the contract was established.", + "c-en-0005": "The matching apparatus according to claim 4 , wherein the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy with consideration given to co-occurrence with the term in the sensor-side metadata or the application-side metadata.", + "c-en-0006": "The matching apparatus according to claim 4 , wherein in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "c-en-0007": "The matching apparatus according to claim 2 , wherein the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy with consideration given to co-occurrence with the term in the sensor-side metadata or the application-side metadata.", + "c-en-0008": "The matching apparatus according to claim 2 , wherein in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "c-en-0009": "The matching apparatus according to claim 2 , wherein the processor is configured with the program to perform operations further comprising: operation as a term database in which a term used in the sensor-side metadata or the application-side metadata, a related term that is related to the term, and a relevancy between the term and the related term are stored in association with each other; and operation as a term database updating unit that updates the term database with use of a term of the sensor-side metadata or the application-side metadata that was newly used in matching, the first or second related term, or the first or second relevancy that was newly calculated.", + "c-en-0010": "The matching apparatus according to claim 3 , wherein the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy with consideration given to co-occurrence with the term in the sensor-side metadata or the application-side metadata.", + "c-en-0011": "The matching apparatus according to claim 3 , wherein in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "c-en-0012": "The matching apparatus according to claim 3 , wherein the processor is configured with the program to perform operations further comprising: operation as a term database in which a term used in the sensor-side metadata or the application-side metadata, a related term that is related to the term, and a relevancy between the term and the related term are stored in association with each other; and operation as a term database updating unit that updates the term database with use of a term of the sensor-side metadata or the application-side metadata that was newly used in matching, the first or second related term, or the first or second relevancy that was newly calculated.", + "c-en-0013": "The matching apparatus according to claim 1 , wherein the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy with consideration given to co-occurrence with the term in the sensor-side metadata or the application-side metadata.", + "c-en-0014": "The matching apparatus according to claim 13 , wherein in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, the processor is configured with the program such that operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "c-en-0015": "The matching apparatus according to claim 1 , wherein the processor is configured with the program such that, in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, operation as the sensor candidate extraction unit comprises correcting the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "c-en-0016": "The matching apparatus according to claim 1 , further comprising: operation as a term database in which a term used in the sensor-side metadata or the application-side metadata, a related term that is related to the term, and a relevancy between the term and the related term are stored in association with each other; and operation as a term database updating unit that updates the term database with use of a term of the sensor-side metadata or the application-side metadata that was newly used in matching, the first or second related term, or the first or second relevancy that was newly calculated.", + "c-en-0017": "The matching apparatus according to claim 1 , wherein the processor is configured with the program to perform operations further comprising: operation as a sensor candidate presentation unit that presents information regarding the extracted candidate sensor to the application-side user.", + "c-en-0018": "A matching method comprising: acquiring sensor-side metadata comprising information regarding a sensor that outputs sensing data; acquiring application-side metadata comprising information regarding an application that provides a service with use of the sensing data; performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; calculating at least either one of a first relevancy between the term in the sensor-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the second related term; with the use of at least either one of the first related term and the first relevancy or the second related term and the second relevancy, performing matching between the sensor-side metadata and the application-side metadata, and extracting the candidate sensor that can provide sensing data that matches needs of the application; and transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "c-en-0019": "A non-transitory computer-readable storage medium storing a program that causes a computer to perform processes comprising: a process of acquiring sensor-side metadata comprising information regarding a sensor that outputs sensing data; a process of acquiring application-side metadata comprising information regarding an application that provides a service with use of the sensing data; a process of performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; a process of calculating at least either one of a first relevancy between the term in the sensor-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the second related term; a process of, with the use of at least either one of the first related term and the first relevancy or the second related term and the second relevancy, performing matching between the sensor-side metadata and the application-side metadata, and extracting the candidate sensor that can provide sensing data that matches needs of the application; and a process of transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0001": "TECHNICAL FIELD", + "p0002": "The present invention relates to technology for performing matching between a sensor side, which provides sensing data detected by sensors, and an application side, which provides services using the sensing data, with use of metadata from the two sides, and for extracting candidate sensors that are capable of providing sensing data that satisfies the needs of an application.", + "p0003": "RELATED ART", + "p0004": "So-called sensor network technology has been under consideration. With this technology, sensor devices that have a sensing function and a communication function are disposed at various locations and on various pieces of industrial equipment, and the sensor devices are networked in order to enable the collection, management, and seamless use of sensing data.", + "p0005": "Normally, a sensor is disposed in order to collect data that is needed by an owner of the sensor. Therefore, when the owner is not collecting data, such a sensor often is not used . The sensing data therefore has low distributability, and regardless of how significant the data is to a third party, analysis and usage of such data stops at the owner of the sensor. This has consequently led to overlapping investment in equipment and network congestion caused by communication with sensors disposed by various owners.", + "p0006": "So-called IoT technology has also been under consideration. With this technology, information regarding many things that exist in the world is combined on the Internet in order to generate new value, thus creating expectation for the seamless expansion of various services in social infrastructure and the like. Producing value from the IoT requires knowledge of the states of the things that are connected to the Internet, and sensing and communication become important component technology.", + "p0007": "Upon forming a sensor network such as that described above, the large amount of sensing data collected around the world needs to be made available to various businesses and organizations according to their own objectives. One example is the processing of data on an application server that is capable of processing big data in order to produce and provide added value, and another example is the vitalization of sensing data transactions to bring about economic effects. For example, the owner of a sensor can gain value by permitting temporary usage of the sensor to a data user or providing sensing data to them. There is also an advantage to the data user in that they can obtain necessary data inexpensively due to not needing investment for installing sensors.", + "p0008": "In view of this, as disclosed in Patent Document 1, an invention has been proposed regarding a mechanism for appropriately distributing a resource such as sensing data.", + "p0009": "In the invention disclosed in Patent Document 1, matching is performed between application-side metadata and sensor-side metadata, and applications that require sensing data are associated with sensors that can provide that data. A dataflow control command is then transmitted to an apparatus that manages a sensor. This therefore promotes the distribution of sensing data in consideration of various conditions and improves the service, and moreover is beneficial to both the data provider and user. Here, \u201cmetadata\u201d refers to information used in server searching and matching, \u201csensor-side metadata\u201d refers to information regarding attributes of a sensor and sensing data obtained by that sensor, and \u201capplication-side metadata\u201d refers to information regarding attributes of an application itself and sensing data required by that application. Also, the dataflow control command is command information that includes information for specifying the sensor that is the data provision source and the application that is the data use destination, and that instructs the distribution of data from the data provision source to the data use destination.", + "p0010": "RELATED ART DOCUMENT", + "p0011": "Patent Document", + "p0012": "Patent Document 1: JP 5445722B", + "p0013": "SUMMARY OF THE INVENTION", + "p0014": "Problem to be Solved by the Invention", + "p0015": "Also, with the invention disclosed in Patent Document 1, there are thought to be many cases where sensor-side metadata and application-side metadata are registered by users, and also many cases of being registered by entrusted persons.", + "p0016": "However, the terms that are registered in the sensor-side metadata and the application-side metadata are not uniform, but rather are diverse to a certain extent. For example, a sensor for measuring the temperature may be named \u201ctemperature sensor\u201d in some cases, and \u201cthermometer\u201d or the like in other cases. In this way, various expressions are conceivable due to variation in expression, synonyms, synonymous terms, generic concepts, specific concepts, and the like.", + "p0017": "Accordingly, if focus is placed on only the registered terms when matching is performed between sensor-side metadata and application-side metadata, there are cases where it is not possible to reflect the data distribution intentions of the sensor-side or application-side user, and an opportunity to match sensor-side metadata with application-side metadata is lost.", + "p0018": "However, if the number of matching opportunities is increased, sensors that are completely unnecessary to the application side will be extracted as matching candidates, thus lowering the precision in matching.", + "p0019": "In view of this, the present invention was achieved in light of the foregoing problems, and an object thereof is to provide technology for increasing the number of opportunities for matching between sensor-side metadata and application-side metadata while also suppressing a reduction in matching precision.", + "p0020": "Means for Solving the Problems", + "p0021": "One aspect of this invention is a matching apparatus including: a sensor-side metadata acquisition unit that acquires sensor-side metadata that is information regarding a sensor that outputs sensing data; an application-side metadata acquisition unit that acquires application-side metadata that is information regarding an application that provides a service with use of the sensing data; a matching unit that performs matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracts a candidate sensor that can provide sensing data that matches needs of the application; and an instruction unit that transmits, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the sensor extracted by the matching unit and the application.", + "p0022": "According to this configuration, it is possible to reflect the data distribution intentions of the sensor-side or application-side user, and also possible to increase the number of opportunities for matching between sensor-side metadata and application-side metadata, and suppress a reduction in matching precision.", + "p0023": "Furthermore, due to being able to increase the number of matching opportunities and also suppress a reduction in matching precision, it is consequently possible to output an appropriate number of dataflow control commands, suppress network communication traffic, and effectively utilize resources in the system overall.", + "p0024": "Also, a configuration is possible in which the matching unit includes a relevancy calculation unit that calculates at least either one of a first relevancy between the term in the sensor-side metadata and the first related term and a second relevancy between the term in the application-side metadata and the second related term, and a sensor candidate extraction unit that, with use of at least either one of the first related term and the first relevancy or the second related term and the second relevancy, performs matching between the sensor-side metadata and the application-side metadata, and extracts the candidate sensor that can provide sensing data that matches needs of the application.", + "p0025": "According to this configuration, relevancy between a term and a related term is introduced, thus making it possible to create an indicator of the extent of matching between the sensor-side metadata and the application-side metadata.", + "p0026": "Also, a configuration is possible in which the sensor candidate extraction unit performs matching between the sensor-side metadata and the application-side metadata with use of the first related term having a first relevancy that is in a predetermined range and the first relevancy, or with use of the second related term having a second relevancy that is in a predetermined range and the second relevancy.", + "p0027": "According to this configuration, matching is performed with use of a related term having a relevancy that is in a predetermined range, thus making it possible to suppress an increase in the number of related terms that are in common between a sensor-side metadata term and an application-side metadata term, and making it possible to perform more precise matching.", + "p0028": "Also, a configuration is possible in which the matching apparatus further includes: a range setting unit that sets the predetermined range of the first relevancy to be used with respect to the first related term, or the predetermined range of the second relevancy to be used with respect to the second related term.", + "p0029": "According to this configuration, the range of relevancy that is to be used can be set to a desired range of relevancy, and the number of related terms that are to be used in matching can be adjusted.", + "p0030": "Also, a configuration is possible in which in a case where a contract is established with an application-side user through matching, the range setting unit sets, as a range of relevancy of a related term to be used in subsequent matching processing, a range of relevancy of a related term that was used when the contract was established.", + "p0031": "According to this configuration, the range of relevancy of a related term that was used when a contract was established is used when subsequent matching processing is performed, and therefore the range of relevancy of a related term can be set to an appropriate range.", + "p0032": "Also, a configuration is possible in which the sensor candidate extraction unit corrects the first relevancy or the second relevancy with consideration given to co-occurrence with the term in the sensor-side metadata or the application-side metadata.", + "p0033": "According to this configuration, terms that co-occur with each other in the sensor-side metadata or the application-side metadata are terms that have a high relevancy with each other, and by correcting the relevancy of a related term with consideration given to such co-occurrence, it is possible to raise the matching precision. Note that co-occurrence of terms can also be said to be an indicator of relevancy of the present invention.", + "p0034": "Also, a configuration is possible in which in a case where the term in the sensor-side metadata and the first related term or the term in the application-side metadata and the second related term were previously used in matching and a contract was established with an application-side user through the matching, the sensor candidate extraction unit corrects the first relevancy or the second relevancy according to a frequency of the contract establishment.", + "p0035": "According to this configuration, among previous matching results, terms that were used when a contract was established can be inferred to have a high relevancy with each other, and therefore by using the frequency of contract establishment when correcting relevancy, it is possible to raise the matching precision. Note that the frequency of contract establishment can also be said to be an indicator of relevancy in the present invention.", + "p0036": "Also, a configuration is possible in which the matching apparatus further includes: a term database in which a term used in the sensor-side metadata or the application-side metadata, a related term that is related to the term, and a relevancy between the term and the related term are stored in association with each other; and a term database updating unit that updates the term database with use of a term of the sensor-side metadata or the application-side metadata that was newly used in matching, the first or second related term, or the first or second relevancy that was newly calculated.", + "p0037": "According to this configuration, by creating a database of calculated relevancy, it is possible to reduce the number of terms for which relevancy is calculated when matching is performed at a later time.", + "p0038": "Also, a configuration is possible in which the matching apparatus further includes: a sensor candidate presentation unit that presents information regarding the extracted candidate sensor to the application-side user.", + "p0039": "According to this configuration, due to presenting the sensor that was extracted as the candidate sensor to the application-side user, the application-side user can check whether or not it is a desired sensor.", + "p0040": "Also, a matching method according to this invention includes: acquiring sensor-side metadata that is information regarding a sensor that outputs sensing data; acquiring application-side metadata that is information regarding an application that provides a service with use of the sensing data; performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; and transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0041": "According to this configuration, it is possible to reflect the data distribution intentions of the user on the sensor side or the application side, and also possible to increase the number of opportunities for matching between sensor-side metadata and application-side metadata, and suppress a reduction in matching precision.", + "p0042": "Furthermore, due to being able to increase the number of matching opportunities and also suppress a reduction in matching precision, it is consequently possible to output an appropriate number of dataflow control commands, suppress network communication traffic, and effectively utilize resources in the system overall.", + "p0043": "Also, a program according to this invention causes a computer to execute: a process of acquiring sensor-side metadata that is information regarding a sensor that outputs sensing data; a process of acquiring application-side metadata that is information regarding an application that provides a service with use of the sensing data; a process of performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; and a process of transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0044": "According to this configuration, it is possible to reflect the data distribution intentions of the user on the sensor side or the application side, and also possible to increase the number of opportunities for matching between sensor-side metadata and application-side metadata, and suppress a reduction in matching precision.", + "p0045": "Furthermore, due to being able to increase the number of matching opportunities and also suppress a reduction in matching precision, it is consequently possible to output an appropriate number of dataflow control commands, suppress network communication traffic, and effectively utilize resources in the system overall.", + "p0046": "Effects of the Invention", + "p0047": "The present invention makes it possible to increase the number of opportunities for matching between sensor-side metadata and application-side metadata, and also makes it possible to suppress a reduction in matching precision.", + "p0048": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0049": "FIG. 1 is a block diagram of a first embodiment.", + "p0050": "FIG. 2 is a block diagram of a matching unit 13 .", + "p0051": "FIG. 3 is a diagram for illustrating operations of a relevancy calculation unit 131 and a sensor candidate extraction unit 132 .", + "p0052": "FIG. 4 is a diagram for illustrating processing performed by the sensor candidate extraction unit 132 .", + "p0053": "FIG. 5 is a block diagram of the matching unit 13 .", + "p0054": "FIG. 6 is a diagram for illustrating operations of the relevancy calculation unit 131 , the sensor candidate extraction unit 132 , and a sensor candidate presentation unit 133 using a specific example in the first embodiment.", + "p0055": "FIG. 7 is a diagram showing an example of presenting sensor candidates to an application-side user.", + "p0056": "FIG. 8 is a block diagram of the matching unit 13 in a modified example of the first embodiment.", + "p0057": "FIG. 9 is a diagram for illustrating a first modified example of the first embodiment.", + "p0058": "FIG. 10 is a diagram for illustrating a second modified example of the first embodiment.", + "p0059": "FIG. 11 is a block diagram of another modified example of the first embodiment.", + "p0060": "FIG. 12 is a block diagram of the matching unit 13 .", + "p0061": "FIG. 13 is a diagram showing an example of a term database 6 registered by a term database updating unit 135 .", + "p0062": "FIG. 14 is a diagram for illustrating operations of the relevancy calculation unit 131 , the sensor candidate extraction unit 132 , and the sensor candidate presentation unit 133 using a specific example in a second embodiment.", + "p0063": "FIG. 15 is a diagram showing an example of presenting sensor candidates to an application-side user based on the example shown in FIG. 14 .", + "p0064": "FIG. 16 is a diagram showing an example of the term database 6 registered by the term database updating unit 135 .", + "p0065": "FIG. 17 is a diagram showing an example of the term database 6 registered by the term database updating unit 135 .", + "p0066": "FIG. 18 is a diagram for illustrating operations of the relevancy calculation unit 131 and the sensor candidate extraction unit 132 in a third embodiment.", + "p0067": "FIG. 19 is a diagram for illustrating operations of the relevancy calculation unit 131 , the sensor candidate extraction unit 132 , and the sensor candidate presentation unit 133 using a specific example in the third embodiment.", + "p0068": "FIG. 20 is a diagram showing an example of presenting sensor candidates to an application-side user based on the example shown in FIG. 19 .", + "p0069": "FIG. 21 is a flowchart of operations in the embodiments.", + "p0070": "FIG. 22 is a flowchart of operations in the embodiments.", + "p0071": "FIG. 23 is another configuration diagram of an embodiment.", + "p0072": "EMBODIMENTS OF THE INVENTION", + "p0073": "Embodiments of the present invention will be described with reference to the drawings.", + "p0074": "In the embodiments of the present invention, in order to effectively use information on things that are connected to a network and constitute an IoT, matching is performed between a sensor side, which provides sensing data detected by sensors, and an application side, which provides a service using the sensing data, with use of metadata from the two sides, and candidate sensors that can provide sensing data that satisfies the needs of an application are extracted. Also, examples will be described in which when a match is found between sensor-side metadata and application-side metadata, sensing data is distributed from the sensor side to the application side.", + "p0075": "First Embodiment", + "p0076": "The following describes a first embodiment of the present invention.", + "p0077": "FIG. 1 is a block diagram of the first embodiment. In FIG. 1, 1 denotes a matching apparatus, 2 denotes a thesaurus, 3 denotes a sensor data providing system, 4 denotes a data distribution control apparatus , and 5 denotes an application system.", + "p0078": "The thesaurus 2 is a systematized thesaurus/dictionary database that classifies terms based on generic-specific relationships, part-whole relationships, synonymous relationships, quasi-synonymous relationships, co-occurrence relationships, and the like between terms . Note that the present embodiment describes an example in which the matching apparatus 1 and the thesaurus 2 are connected via network, but the matching apparatus 1 may include the thesaurus 2 .", + "p0079": "The sensor data providing system 3 is a system that provides sensing data acquired by sensors.", + "p0080": "The data distribution control apparatus 4 has functions for receiving sensing data from the sensor data providing system 3 and distributing sensing data to the application system 5 , in accordance with later-described dataflow control commands.", + "p0081": "The application system 5 is a system that provides a service using sensing data provided by the sensor data providing system 4 .", + "p0082": "In the example described below, the matching apparatus 1 , the sensor data providing system 3 , the data distribution control apparatus 4 , and the application system 5 are connected via a network.", + "p0083": "The matching apparatus 1 includes a sensor-side metadata acquisition unit 11 , an application-side metadata acquisition unit 12 , a matching unit 13 , and a dataflow control command instruction unit 14 .", + "p0084": "The sensor-side metadata acquisition unit 11 acquires sensor-side metadata, which is information regarding a sensor that outputs sensing data.", + "p0085": "Here, a sensor is a device that detects some sort of physical quantity and change thereof, and records or outputs such data as sensing data. Typical examples of such a sensor include a position sensor , an acceleration sensor, and a pressure sensor, but a camera, a microphone, an input system, and the like can also be said to be sensors.", + "p0086": "Sensor-side metadata refers to information regarding attributes of a sensor and sensing data obtained by that sensor, and is for use in later-described matching. Examples of sensor-side metadata items include a data category, a data item, a measurement region, a measurement date/time, a device, a distribution interval, and a data specification. For example, \u201cindoor environment measurement\u201d is described as the data category, \u201ctemperature, humidity, volume, acceleration\u201d is described as the data item, \u201cShiga Prefecture . . . \u201d is described as the measurement region, \u201cimmediate\u201d is described as the measurement date/time, \u201cdevice name\u201d is described as the device, \u201cOnce/10 minutes\u201d is described as the distribution interval, and \u201chttp://WWW.XXX\u201d is described as the data specification. In the following description, it is assumed that the words such as nouns and numerical counts described as items in the sensor-side metadata are simply described as sensor-side metadata terms. Note when there is a group of words, such as the case of a geographical name, there are cases where morphological analysis or the like is used to divide the group of words into words that have respective meanings for usage.", + "p0087": "Also, the sensor-side metadata acquisition unit 11 can have various configurations. For example, there is a method of storing the aforementioned metadata in a database that can be accessed from the matching apparatus 1 . In this case, metadata necessary for matching processing is already in the database, and therefore it is sufficient that the trigger for acquisition is an event notification given from the sensor side or the application side. Furthermore, there is also a configuration in which a sensor-side metadata database does not exist. In this case, metadata is transmitted from an application or a sensor management apparatus that detected the generation of an event.", + "p0088": "The application-side metadata acquisition unit 12 acquires application-side metadata, which is information regarding applications that provide services using sensing data.", + "p0089": "Here, application-side metadata refers to information regarding the attributes of an application itself and sensing data needed by that application. Examples of application-side metadata items include a data category, a data item, measurement region, a measurement date/time, and a device. For example, \u201cenvironment measurement\u201d is described as the data category, \u201ctemperature, humidity\u201d is described as the data item, \u201cShiga Prefecture . . . \u201d is described as the measurement region, \u201cimmediate\u201d is described as the measurement date/time, and \u201cdevice name\u201d is described as the device. In the following description, it is assumed that the words such as nouns and numerical counts described as items in the application-side metadata are simply described as application-side metadata terms. Note when there is a group of words, such as the case of a geographical name, there are cases where morphological analysis or the like is used to divide the group of words into words that have respective meanings for usage.", + "p0090": "The application-side metadata acquisition unit 12 can have various configurations. For example, there is a method of storing the aforementioned metadata in a database that can be accessed from the matching apparatus 1 . In this case, metadata necessary for matching processing is already in the database, and therefore it is sufficient that the trigger for acquisition is an event notification given from the sensor side or the application side. Furthermore, there is also a configuration in which an application-side metadata database does not exist. In this case, the application-side user acquires application-side metadata that satisfies the needs of an application and that is created when searching for a desired sensor.", + "p0091": "The matching unit 13 performs matching between sensor-side metadata and application-side metadata in which a related term that is related to a term in acquired sensor-side metadata and a related term that is related to a term in acquired application-side metadata are in common with each other, and extracts candidate sensors that can provide sensing data that satisfies the needs of an application. Note that related term refers to a term that is a more generic or specific concept of a term in sensor-side metadata or application-side metadata, is in a synonymous relationship therewith, or is in a quasi-synonymous relationship therewith, for example.", + "p0092": "The matching unit 13 searches the thesaurus 2 for related terms that are related to the terms in the items of the sensor-side metadata. The matching unit 13 similarly searches the thesaurus 2 for related terms that are related to the terms in the items of the application-side metadata. If the found related terms include a common related term in both the sensor-side metadata and the application-side metadata, matching is performed between the sensor-side metadata and the application-side metadata that correspond to the common related term, and candidate sensors that can provide sensing data that satisfies the needs of an application are extracted.", + "p0093": "It is not often the case that terms in the metadata registered by the sensor-side and application-side user match completely. However, the application-side user does not necessarily desire only a sensor that completely matches a term registered by the user, and it is thought that there are often cases where a sensor represented by a term that is a more generic or specific concept of the registered metadata term, in a synonymous relationship therewith, or in a quasi-synonymous relationship therewith for example will satisfy the application-side user's needs.", + "p0094": "In view of this, in the present embodiment, the matching unit 13 extracts, as candidates, sensors that have a possibility of satisfying the application-side user's needs, and therefore the sensor-side metadata and the application-side metadata that are used in matching is extended to sensor-side metadata and application-side metadata in which a related term that is related to a sensor-side metadata term and a related term that is related to an application-side metadata term are in common with each other, thus increasing the number of matching opportunities while also suppressing a reduction in matching precision.", + "p0095": "Also, when searching for related terms that are related to a sensor-side metadata term and related terms that are related to an application-side metadata term, if even related terms having low relevancy are used, the number of related terms that are in common with the sensor-side metadata term and the application-side metadata term increases, thus not only increasing the processing load, but also having a risk of reducing the matching precision. In view of this, if it is possible to know to the extent to which a sensor-side metadata term or an application-side metadata term is related to a related term, and to limit the related terms that are used, then matching can be performed with higher precision.", + "p0096": "In view of this, in the first embodiment, the concept of relevancy of a related term is introduced to the matching unit 13 . FIG. 2 is a block diagram of the matching unit 13 .", + "p0097": "As shown in FIG. 2 , the matching unit 13 includes a relevancy calculation unit 131 and a sensor candidate extraction unit 132 . Hereinafter, the relevancy calculation unit 131 and the sensor candidate extraction unit 132 will be described using an example shown in FIG. 3 .", + "p0098": "The relevancy calculation unit 131 calculates the relevancy between terms in the items of the sensor-side metadata and the related terms registered in the thesaurus 2 , and the relevancy between terms in the items of the application-side metadata and the related terms registered in the thesaurus 2 .", + "p0099": "Here, relevancy refers to how similar or related terms are to each other. The relevancy calculated by the relevancy calculation unit 131 can be calculated using any type of method or any conventional method, as long as it is possible to calculate an extent indicating how similar or related terms are to each other. One example of a conventional method is the relevancy calculation method described in the document \u201cAutomatic Extraction of Related terms using Web Search Engines, by Keigo Watanabe, Danushka Bollegala, Yutaka Matsuo, and Mitsuru Ishizuka\u201d. Also, another relevancy calculation method is a technique in which the positions of terms in a word space are represented as distances. According to this method, the smaller the distance is between terms, the higher the relevancy between the terms is, and the larger the distance is between terms, the lower the relevancy between the terms is.", + "p0100": "In the following description of the relevancy calculation unit 131 , from the viewpoint of generic-specific concepts, a synonymous relationship, a quasi-synonymous relationship, and a co-occurrence relationship, the relevancy between terms will be described using the example of a dissimilarity F calculated as a value indicating the extent to which terms are different from each other. The dissimilarity F takes a smaller value the less dissimilar terms are, and takes a higher value the more dissimilar terms are. Accordingly, the smaller the dissimilarity F value is, the higher the relevancy between terms is, and the higher the dissimilarity F value is, the lower the relevancy between terms is. Note that a conventional method can be used to calculate the dissimilarity F, and therefore a specific calculation method will not be described, and only calculated values for the dissimilarity F will be described.", + "p0101": "First, for each term in the sensor-side metadata items, the terms registered in the thesaurus 2 are searched to find related terms that are related to that term. For example, for each term in the sensor-side metadata items, the terms registered in the thesaurus 2 are searched to find first related terms, which are terms that are a more generic or specific concept of that term, are in a synonymous relationship or a quasi-synonymous relationship therewith, or have a high frequency of co-occurrence therewith, for example.", + "p0102": "Next, the dissimilarity F is calculated for each related term for the sensor-side metadata term. In the following, it is assumed that there are related terms a, b, c, and g for a term X in the sensor-side metadata, and the dissimilarities of the related terms a, b, c, and g for the term X in the sensor-side metadata will be represented by \u201cdissimilarity F \u201d. The smaller the value of dissimilarity F is, the higher the relevancy is, and the larger the value of dissimilarity F is, the lower the relevancy is. Here, the dissimilarity F takes the following values for the related terms a, b, c, and g for the term X in the sensor-side metadata.", + "p0103": "dissimilarity F =0.80", + "p0104": "dissimilarity F =0.60", + "p0105": "dissimilarity F =1.20", + "p0106": "dissimilarity F =0.90", + "p0107": "Using a similar method, for each term in the application-side metadata items, the terms registered in the thesaurus 2 are searched to find related terms that are related to that term. For example, for each term in the application-side metadata items, the terms registered in the thesaurus 2 are searched to find second related terms, which are terms that are a more generic or specific concept of that term, are in a synonymous relationship or a quasi-synonymous relationship therewith, or high have a high frequency of co-occurrence therewith, for example.", + "p0108": "Next, the dissimilarity F is calculated for each related term for the application-side metadata term. In the following, it is assumed that there are related terms a, d, e, f, and g for a term Y in the application-side metadata, and the dissimilarities F of the related terms a, d, e, f, and g for the term Y in the application-side metadata will be represented by \u201cF \u201d. Here, the dissimilarity F takes the following values for the related terms a, d, e, f, and g for the term Y in the application-side metadata.", + "p0109": "dissimilarity F =0.60", + "p0110": "dissimilarity F =0.80", + "p0111": "dissimilarity F =0.60", + "p0112": "dissimilarity F =1.20", + "p0113": "dissimilarity F =1.10", + "p0114": "Next, the sensor candidate extraction unit 132 will be described.", + "p0115": "The sensor candidate extraction unit 132 performs matching between the sensor-side metadata and the application-side metadata based on the dissimilarities F that were calculated by the relevancy calculation unit 131 . The sensor-side metadata and the application-side metadata that are used in matching are sensor-side metadata and application-side metadata in which an acquired related term that is related to a term in the sensor-side metadata and an acquired related term that is related to a term in the application-side metadata are in common with each other. Specifically, they are sensor-side metadata and application-side metadata that both include the same related term having a dissimilarity F value. Matching is performed using the related terms and the dissimilarities F in such sensor-side metadata and application-side metadata.", + "p0116": "Next, the extraction of sensor candidates by the sensor candidate extraction unit 132 will be described. In the following description, it is assumed that the relevancy calculation unit 131 has calculated dissimilarities F as described above.", + "p0117": "Here, all of the related terms for which the dissimilarity F was calculated may be used in matching, but this increases the number of related terms and increases the matching processing load, and furthermore, the use of related terms that have low relevancy leads to the possibility of performing matching that is not intended by the user. In view of this, related terms having a dissimilarity F that is lower than a predetermined threshold value are used in metadata matching. Specifically, a related term having a dissimilarity F that is lower than the predetermined threshold value is determined to have a low dissimilarity and a high relevancy, whereas a related term having a dissimilarity F that is greater than or equal to the predetermined threshold value is determined to have a high dissimilarity and a low relevancy, and related terms having a high relevancy are used in matching.", + "p0118": "FIG. 3 shows an example of the case where the threshold value for determining which related terms are to be used is 1. Specifically, the related terms for which the dissimilarity F with the term X in the sensor-side metadata item is less than 1 are the related term a, the related term b, and the related term g. Accordingly, the first related terms that will be used as related term for the term X in the sensor-side metadata are the related term a, the related term b, and the related term g. On the other hand, the related term c has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0119": "Also, the related terms for which the dissimilarity F with the term Y in the application-side metadata item is less than 1 are the related term a, the related term d, and the related term e. Accordingly, the related terms that will be used as second related terms for the term X in the application-side metadata are the related term a, the related term d, and the related term e. On the other hand, the related term g and the related term g have a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0120": "Note that although the threshold value for the dissimilarity F for use as a related term is the same for both the sensor-side metadata and the application-side metadata in the above example, different threshold values may be set.", + "p0121": "Next, the sensor candidate extraction unit 132 determines that the sensor-side metadata and the application-side metadata match if the first related term for the term in the sensor-side metadata and the second related term for the term in the application-side metadata are in common with each other, and extracts the sensor corresponding to that sensor-side metadata as a sensor candidate.", + "p0122": "In the example in FIG. 3 , the first related terms for the term X in the sensor-side metadata are the related term a, the related term b, and the related term g. Also, the second related terms for the term Y in the application-side metadata item are the related term a, the related term d, and the related term e. The related term a is a common related term. Accordingly, it is determined that there is a match between the sensor-side metadata and the application-side metadata that have the related term a in common, and the sensor corresponding to the sensor-side metadata having the term X is extracted as a sensor candidate.", + "p0123": "The processing of the sensor candidate extraction unit 132 described above is shown in FIG. 4 .", + "p0124": "In the example in FIG. 4 , on the left side, the dissimilarity F values between the related terms and the term X in the sensor-side metadata item are arranged in descending order, and on the right side, the dissimilarity F values between the related terms and the term Y in the application-side metadata item are arranged in descending order. This also shows an example in which the related terms having a dissimilarity F less than the threshold value 1 are used in matching.", + "p0125": "Specifically, the related terms for which the dissimilarity F with the term X in the sensor-side metadata item is less than 1 are the related term a, the related term b, and the related term g. Accordingly, the first related terms that will be used as related term for the term X in the sensor-side metadata are the related term a, the related term b, and the related term g. On the other hand, the related term c has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0126": "Also, the related terms for the term Y in the application-side metadata item that have a dissimilarity F less than 1 are the related term a, the related term d, and the related term e. Accordingly, the related terms that will be used as second related terms for the term X in the application-side metadata are the related term a, the related term d, and the related term e. On the other hand, the related term f and the related term g have a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0127": "The related term a is a common related term. Accordingly, it is determined that there is a match between the sensor-side metadata and the application-side metadata that have the related term a in common, and the sensor corresponding to the sensor-side metadata having the term X is extracted as a sensor candidate.", + "p0128": "A configuration is possible in which, at this time, the matching unit 13 presents the sensor extracted as a sensor candidate to the application-side user. FIG. 5 is a block diagram of the matching unit 13 in the case of including a function for presenting, to the application-side user, a candidate sensor that is capable of providing sensing data that satisfies the needs of the matching application.", + "p0129": "As shown in FIG. 5 , the matching unit 13 includes a sensor candidate presentation unit 133 . The sensor candidate presentation unit 133 presents, to the application-side user, a sensor candidate obtained from the sensor candidate extraction unit 132 and the metadata of that sensor. The present method may be a method of displaying the sensor candidate and the metadata of that sensor on a display, a method of printing out a document describing that content, or sending the user an e-mail describing that content.", + "p0130": "Next, a specific example of operations of the relevancy calculation unit 131 , the sensor candidate extraction unit 132 , and the sensor candidate presentation unit 133 will be described with reference to FIG. 6 .", + "p0131": "In the following description, the sensors include an \u201cabsolute pressure sensor\u201d, a \u201cblood pressure meter\u201d, and a \u201cload sensor\u201d, and the sensor-side metadata includes the term \u201cabsolute pressure sensor\u201d in sensor-side metadata 1 for the \u201cabsolute pressure sensor\u201d, the term \u201cblood pressure meter\u201d in sensor-side metadata 2 for the \u201cblood pressure meter\u201d, and the term \u201cload sensor\u201d in sensor-side metadata 3 for the \u201cload sensor\u201d. Also, the term \u201cpressure sensor\u201d is included as application-side metadata.", + "p0132": "First, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 . Here, the related term \u201cair pressure sensor\u201d and the related term \u201ctemperature sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 . The dissimilarity F is then calculated between the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 and each of the related terms. The calculation results are as follows.", + "p0133": "dissimilarity F =0.20", + "p0134": "dissimilarity F =0.40", + "p0135": "Next, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 . Here, the related term \u201cpressure sensor\u201d, the related term \u201cblood pressure monitor\u201d, and the related term \u201cblood pressure sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 . The dissimilarity F is then calculated between the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 and each of the related terms. The calculation results are as follows.", + "p0136": "dissimilarity F =0.50", + "p0137": "dissimilarity F =0.10", + "p0138": "dissimilarity F =0.10", + "p0139": "Furthermore, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cload sensor\u201d in the sensor-side metadata 3 . Here, the related term \u201cstrain gauge\u201d and the related term \u201cweight sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cload sensor\u201d in the sensor-side metadata 3 . The dissimilarity F is then calculated between the term \u201cload sensor\u201d in the sensor-side metadata 3 and each of the related terms. The calculation results are as follows.", + "p0140": "dissimilarity F =0.30", + "p0141": "dissimilarity F =0.10", + "p0142": "Furthermore, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cpressure sensor\u201d in the application-side metadata. Here, the related term \u201cpressure meter\u201d, the related term \u201cpressure gauge\u201d, the related term \u201cpressure monitor\u201d, the related term \u201cair pressure sensor\u201d, the related term \u201cblood pressure meter\u201d, and the related term \u201cstrain gauge\u201d are found in the thesaurus 2 as related terms for the term \u201cpressure sensor\u201d in the application-side metadata. The dissimilarity F is then calculated between the term \u201cpressure sensor\u201d in the application-side metadata and each of the related terms. The calculation results are as follows.", + "p0143": "dissimilarity F =0.10", + "p0144": "dissimilarity F =0.10", + "p0145": "dissimilarity F =0.10", + "p0146": "dissimilarity F =0.70", + "p0147": "dissimilarity F =0.50", + "p0148": "dissimilarity F =1.20", + "p0149": "The sensor candidate extraction unit 132 then performs matching using, among the related terms, the related terms that are related to both the sensor-side metadata and the application-side metadata and the dissimilarity F values thereof. At this time, only related terms having a dissimilarity F less than 1 are used in matching.", + "p0150": "Specifically, the related terms that have a dissimilarity F less than 1 for the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 are the related term \u201cair pressure sensor\u201d and the related term \u201ctemperature sensor\u201d. Also, the related terms that have a dissimilarity F less than 1 for the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 are the related term \u201cpressure sensor\u201d, the related term \u201cblood pressure monitor\u201d, and the related term \u201cblood pressure sensor\u201d. Also, the related terms that have a dissimilarity F less than 1 for the term \u201cload sensor\u201d in the sensor-side metadata 3 are the related term \u201cstrain gauge\u201d and the related term \u201cweight sensor\u201d. Moreover, the related terms that have a dissimilarity F less than 1 for the term \u201cpressure sensor\u201d in the application-side metadata are the related term \u201cpressure meter\u201d, the related term \u201cpressure gauge\u201d, the related term \u201cpressure monitor\u201d, the related term \u201cair pressure sensor\u201d, and the related term \u201cblood pressure meter\u201d.", + "p0151": "Among these related terms, the common related terms for the sensor-side metadata and the application-side metadata are the related term \u201cair pressure sensor\u201d and the related term \u201cblood pressure meter\u201d.", + "p0152": "Accordingly, the sensor candidate extraction unit 132 extracts \u201cblood pressure meter\u201d and \u201cabsolute pressure sensor\u201d as candidate sensors that can provide sensing data that satisfies the application-side needs.", + "p0153": "At this time, a priority ranking may be applied to the extracted \u201cblood pressure meter\u201d and \u201cabsolute pressure sensor\u201d. One method of calculating the priority ranking is a method of using a sum value of the dissimilarity F between the common related term and the sensor-side metadata and the dissimilarity F between the common related term and the application-side metadata. This is because it is thought that the smaller the sum value of the dissimilarity F values is, the less dissimilarity there is between the terms. The sum values for \u201cblood pressure meter\u201d and \u201cabsolute pressure sensor\u201d are as shown below.", + "p0154": "- \u201cblood pressure meter\u201d + =dissimilarity F =0.5", + "p0155": "Note that in this example, the sensor-side metadata 2 term and the application-side metadata term are related terms with respect to each other, and therefore the sum value is obtained using the dissimilarity F values for the sensor-side metadata 2 term and the application-side metadata term.", + "p0156": "- \u201cabsolute pressure sensor\u201d + =dissimilarity F +dissimilarity F =0.20+0.70 =0.90", + "p0157": "Accordingly, the sensor candidate extraction unit 132 sets \u201cblood pressure meter\u201d as priority rank 1 and sets \u201cabsolute pressure sensor\u201d as priority rank 2 .", + "p0158": "The sensor candidate presentation unit 133 then presents the sensor candidates to the application-side user. FIG. 7 shows an example of presenting sensor candidates to the application-side user. In FIG. 7 , \u201cSensor A\u201d corresponding to \u201cblood pressure meter\u201d in the sensor-side metadata and \u201cSensor B\u201d corresponding to \u201cabsolute pressure sensor\u201d in the sensor-side metadata, which were extracted by the sensor candidate extraction unit 132 , are presented in a list along with the priority ranks and the dissimilarity F values thereof.", + "p0159": "The dataflow control command instruction unit 14 receives the matching results obtained by the matching unit 13 , and transmits a dataflow control command to the data distribution control apparatus 4 . This dataflow control command is command information that includes information specifying the sensor data providing system 3 that is the data provision source and the application system 5 that is the data use destination, and that instructs the distribution of data from the data provision source to the data use destination.", + "p0160": "For example, in the above-described example in FIG. 6 , in the case where the application system 5 side has designated the sensor \u201cblood pressure meter\u201d having the priority rank 1 , the dataflow control command is command information that includes information for specifying the sensor data providing system 3 , which is to provide sensing data from the sensor \u201cblood pressure meter\u201d, and the application system 5 , which is to use the sensing data from the sensor \u201cblood pressure meter\u201d and has \u201cpressure sensor\u201d registered as metadata, and is command information that instructs the distribution of sensing data obtained by the sensor \u201cblood pressure meter\u201d from the sensor data providing system 3 to the application system 5 .", + "p0161": "Note that the information for specifying the sensor data providing system 3 and the application system 5 that is to be the data use destination may be IP addresses or the like. Also, the dataflow control command can instruct the distribution of sensing data obtained by one sensor to a plurality of applications. It can also instruct the distribution of sensing data from a plurality of sensors to one application. It can furthermore instruct the distribution of sensing data from a plurality of sensors to a plurality of applications.", + "p0162": "Note that the following cases are examples of triggers for the output of the dataflow control command by the dataflow control command instruction unit 14 .", + "p0163": "Case where the matching unit 13 determines, based on a predetermined condition, that there is a match between sensor-side metadata and application-side metadata", + "p0164": "Case where the sensor candidate presentation unit 133 of the matching unit 13 presents, to the application-side user, a matching result between sensor-side metadata and application-side metadata obtained, and the application-side user desires, based on the matching result, the provision of sensing data", + "p0165": "The above cases are merely examples, and there is no limitation to the above cases.", + "p0166": "In the first embodiment described above, the matching data is extended to sensor-side metadata and application-side metadata that have a common related term, thus making it possible to increase the number of matching opportunities while also suppressing a reduction in matching precision. Accordingly, an appropriate number of matches is obtained while also ensuring precision, thus consequently making it possible to output an appropriate number of dataflow control commands, to suppress network communication traffic, and to also effectively utilize resources in the system overall.", + "p0167": "Furthermore, related terms having a high relevancy are used in matching, thus raising the possibility of performing matching that reflects the intentions of the sensor-side or application-side user. Also, the number of referenced related terms decreases, thus making it possible to also reduce the processing burden required for matching.", + "p0168": "Also, according to the first embodiment, matching is performed between application-side metadata and sensor-side metadata, and matching is performed between an application that needs sensing data and a sensor that can provide that data. A dataflow control command is then transmitted to an apparatus that manages the sensor. This therefore promotes the distribution of sensing data in consideration of various conditions and improves the service, and moreover is beneficial to both the data provider and user.", + "p0169": "Note that the above first embodiment describes an example in which both the relevancy between a sensor-side metadata term and a related term and the relevancy between an application-side metadata term and a related term are calculated, but a configuration is possible in which only one relevancy is calculated. For example, in the above-described specific example in FIG. 6 , it can also be said that the sensor-side metadata 2 term \u201cblood pressure meter\u201d and the application-side metadata term \u201cpressure sensor\u201d are related terms that are in common with each other, and in this case, it is possible to calculate the relevancy for only one.", + "p0170": "Also, the above description is given using an example in which the dissimilarity F value decreases as the relevancy between two terms increases, but depending on the method for calculating the dissimilarity F, there are also cases where the dissimilarity F value increases as the relevancy between two terms increases. If this type of method for calculating the dissimilarity F is applied to the present invention, the threshold value used when selecting related terms for use in matching of course also changes, and it goes without saying that a related term having a dissimilarity F value that is greater than or equal to the threshold value or exceeds the threshold value will be selected instead of a related term having a dissimilarity F value that is less than the threshold value.", + "p0171": "Furthermore, the dissimilarity F is one example for obtaining the relevancy between terms, and as long as the relevancy between terms can be obtained, there is no limitation to using the dissimilarity F, and other methods may be used as described above.", + "p0172": "Next, an example of changing the range of relevancy when determining related terms for use in matching will be described as a modified example of the first embodiment.", + "p0173": "First Modified Example of First Embodiment", + "p0174": "In the example described above, the sensor candidate extraction unit 132 sets in advance a threshold value for the dissimilarity F for selecting related terms that are to be used. However, there are cases where a related term for use in matching cannot be extracted according to the set threshold value. In other words, there are cases where the range of relevancy of related terms to be used in matching is too narrow.", + "p0175": "In view of this, as shown in FIG. 8 , the matching unit 13 is provided with a relevancy threshold value setting unit 134 that changes the threshold value for the dissimilarity F for selecting related terms to be used in matching. In order to widen the range of relevancy of the related terms to be used in matching, the relevancy threshold value setting unit 134 raises the threshold value for the dissimilarity F that is set in the sensor candidate extraction unit 132 .", + "p0176": "FIG. 9 is a diagram showing how the related terms used in matching change when the relevancy threshold value setting unit 134 changes the threshold value.", + "p0177": "In FIG. 9 , on the left side, the dissimilarity F values between the related terms and the term X in the sensor-side metadata item are arranged in descending order, and on the right side, the dissimilarity F values between the related terms and the term Y in the application-side metadata item are arranged in descending order. This also shows an example in which the related terms having a dissimilarity F less than the threshold value 1 are used in matching.", + "p0178": "In the example in FIG. 9 , the related terms a, b, c, and g are related to the term X in the sensor-side metadata, and the related terms a, b, c, and g for the term X in the sensor-side metadata have the following dissimilarity F values.", + "p0179": "dissimilarity F =1.10", + "p0180": "dissimilarity F =0.60", + "p0181": "dissimilarity F =1.30", + "p0182": "dissimilarity F =0.60", + "p0183": "Also, the related terms a, d, e, f, and g are related to the term Y in the application-side metadata, and the related terms a, d, e, f, and g for the term Y in the application-side metadata have the following dissimilarity F values.", + "p0184": "dissimilarity F =0.60", + "p0185": "dissimilarity F =0.80", + "p0186": "dissimilarity F =0.60", + "p0187": "dissimilarity F =1.25", + "p0188": "dissimilarity F =1.25", + "p0189": "Accordingly, the related terms for which the dissimilarity F with the term X in the sensor-side metadata is less than 1 are the related term a, the related term b, and the related term g. Accordingly, the first related terms that will be used as related terms for the term X in the sensor-side metadata are the related term a, the related term b, and the related term g. On the other hand, the related term c has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0190": "Also, the related terms for the term Y in the application-side metadata that have a dissimilarity F less than 1 are the related term a, the related term d, and the related term e. Accordingly, the related terms that will be used as second related terms for the term X in the application-side metadata are the related term a, the related term d, and the related term e. On the other hand, the related term g and the related term g have a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0191": "Accordingly, there is no related term that has a dissimilarity F less than 1 and is in common with both the term X in the sensor-side metadata item and the term Y in the application-side metadata item.", + "p0192": "In view of this, the threshold value for the dissimilarity F is changed from 1 to 1.2. The case of changing the threshold value for the dissimilarity F from 1 to 1.2 is shown on the lower side in FIG. 9 .", + "p0193": "Due to changing the threshold value for the dissimilarity F from 1 to 1.2, the related term a that is related to the term X in the sensor-side metadata item and has a dissimilarity F of 1.10 is newly included as a related term that is to be used in matching.", + "p0194": "As a result, the related term a is a related term that is in common between the sensor-side metadata and the application-side metadata. Accordingly, it is determined that there is a match between the sensor-side metadata and the application-side metadata that have the related term a in common, and the sensor corresponding to the sensor-side metadata having the term X is extracted as a sensor candidate.", + "p0195": "Accordingly, a related term for use in matching is more likely to appear. Note that as a variation of the first modified example of the first embodiment, a configuration is also possible in which the application-side user can operate the relevancy threshold value setting unit 134 , and the application-side user increases the threshold value for the dissimilarity F in order to widen the range of relevancy.", + "p0196": "Note that although the dissimilarity F is used as an example of relevancy in the example described above, there is no limitation to this.", + "p0197": "Second Modified Example of First Embodiment", + "p0198": "There are cases where too many related terms for use in matching are selected according to the threshold value set in the relevancy calculation unit 131 . In other words, there are cases where the range of relevancy of related terms to be used in matching is too wide. In view of this, in such a case, in order to set a more narrow range for the relevancy of the related term, the relevancy threshold value setting unit 134 reduces the threshold value for the dissimilarity F for determining the range of relevancy set in the sensor candidate extraction unit 132 , thus obtaining an appropriate number of related terms.", + "p0199": "FIG. 10 is a diagram showing how the related terms used in matching change when the relevancy threshold value setting unit 134 changes the threshold value.", + "p0200": "In FIG. 10 , the dissimilarity F values between the related terms and the term X in the sensor-side metadata item, and the dissimilarity F values with respect to the term M in application A-side metadata item, the term N in application B-side metadata item, and the term O in application C-side metadata item are arranged in descending order, from the left side toward the right side. This also shows an example in which the related terms having a dissimilarity F less than the threshold value 1 are used in matching.", + "p0201": "In the example in FIG. 10 , the related terms a, b, c, d, and f are related to the term X in the sensor-side metadata, and the related terms a, b, c, d, and f for the term X in the sensor-side metadata have the following dissimilarity F values.", + "p0202": "dissimilarity F =1.20", + "p0203": "dissimilarity F =0.90", + "p0204": "dissimilarity F =0.60", + "p0205": "dissimilarity F =0.40", + "p0206": "dissimilarity F =0.30", + "p0207": "Also, the related terms b, d, f, and r are related to the term M in the application A-side metadata, and the related terms b, d, f, and r for the term M in the application A-side metadata have the following dissimilarity F values.", + "p0208": "dissimilarity F =0.40", + "p0209": "dissimilarity F =0.40", + "p0210": "dissimilarity F =1.10", + "p0211": "dissimilarity F =0.60", + "p0212": "Also, the related terms b, d, and q are related to the term N in the application B-side metadata, and the related terms b, d, and q for the term N in the application B-side metadata have the following dissimilarity F values.", + "p0213": "dissimilarity F =0.80", + "p0214": "dissimilarity F =0.50", + "p0215": "dissimilarity F =1.10", + "p0216": "Also, the related terms c, f, and w are related to the term O in the application C-side metadata, and the related terms b, d, and q for the term O in the application C-side metadata have the following dissimilarity F values.", + "p0217": "dissimilarity F =0.40", + "p0218": "dissimilarity F =0.80", + "p0219": "dissimilarity F =1.20", + "p0220": "Accordingly, the related terms for which the dissimilarity F with the term X in the sensor-side metadata is less than 1 are the related term b, the related term c, the related term d, the related term f, and the related term r. Accordingly, the first related terms that will be used as related term for the term X in the sensor-side metadata are the related term b, the related term c, the related term d, the related term f, and the related term r. On the other hand, the related term a has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0221": "Also, the related terms for which the dissimilarity F with the term M in the application A-side metadata is less than 1 are the related term b, the related term d, and the related term r. Accordingly, the related terms that will be used as second related terms for the term M in the application A-side metadata are the related term b, the related term d, and the related term r. On the other hand, the related term f has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0222": "Also, the related terms for which the dissimilarity F with the term N in the application B-side metadata is less than 1 are the related term b and the related term d. Accordingly, the related terms that will be used as second related terms for the term N in the application B-side metadata are the related term b and the related term d. On the other hand, the related term g has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0223": "Also, the related terms for the term O in the application C-side metadata that have a dissimilarity F less than 1 are the related term c and the related term f. Accordingly, the related terms that will be used as second related terms for the term O in the application C-side metadata are the related term c and the related term f. On the other hand, the related term w has a dissimilarity F that is greater than or equal to 1, and will not be used in matching.", + "p0224": "Here, the related terms that are in common between the sensor-side metadata and the application-side metadata are as follows.", + "p0225": "- related term b in common between sensor-side metadata and application B-side metadata - related term b in common between sensor-side metadata and application A-side metadata - related term c in common between sensor-side metadata and application C-side metadata - related term d in common between sensor-side metadata and application A-side metadata - related term d in common between sensor-side metadata and application B-side metadata - related term f in common between sensor-side metadata and application C-side metadata", + "p0226": "In this way, the matching sensor-side metadata is matched with all of the application-side metadata. This is because the range of relevancy of related terms to be used in matching is too wide, and there are too many related terms.", + "p0227": "In view of this, the threshold value for the dissimilarity F is changed from 1 to 0.5. The case of changing the threshold value for the dissimilarity F from 1 to 0.5 is shown on the lower side in FIG. 10 .", + "p0228": "Due to changing the threshold value for the dissimilarity F from 1 to 0.5, the related terms to be used in matching are reduced to the related term b, the related term d, and the related term f. As a result, only the related term d is a related term that is in common between the sensor-side metadata and the application-side metadata.", + "p0229": "Accordingly, it is determined that there is a match between the sensor-side metadata and the application A-side metadata that have the related term d in common, and the sensor corresponding to the sensor-side metadata having the term M is extracted as a sensor candidate.", + "p0230": "In this way, by changing the threshold value, it is possible to set a narrower range for the relevancy of related terms, reduce the number of related terms that are to be used in matching, and raise the matching precision.", + "p0231": "Note that as a variation of the second modified example of the first embodiment, a configuration is also possible in which the application-side user can operate the relevancy threshold value setting unit 134 , the application-side user reduces the threshold value for the dissimilarity F in order to set a narrower range of relevancy.", + "p0232": "Also, although the dissimilarity F is used as an example of relevancy in the example described above, there is no limitation to this.", + "p0233": "Third Modified Example of First Embodiment", + "p0234": "The width of the range of relevancy of related terms is set by the relevancy threshold value setting unit 134 in order for metadata intended by the user to be used in matching. Accordingly, as one method for setting the threshold value that determines the range of relevancy of related terms that are to be used, there is a method in which the relevancy threshold value setting unit 134 stores a threshold value that is set at the time when a contract is established for providing sensor data to an application through matching between sensor-side metadata and application-side metadata, and when matching processing is to be started the next time, the stored threshold value is set as the initial threshold value for the dissimilarity F for selecting related terms that are to be used. This is because when a match is made between sensor-side metadata and application-side metadata, it is thought that the relevancy range of relevancy is appropriate at that time. In other words, it is thought that the threshold value for the dissimilarity F for determining the range of relevancy is appropriate.", + "p0235": "The following cases are examples regarding times when a contract is established for providing sensor data to an application.", + "p0236": "- When a contract for providing sensor sensing data to an application is established by a sensor-side user and an application-side user after matching - When a matching result is received, and then a dataflow control command is transmitted - When a data distribution apparatus having a sensing data distributing function receives a dataflow control command, and sensing data is distributed from a sensor to an application - When a data-using application receives sensing data - When a data-using application has received sensing data, and then it has been checked whether or not the received sensing data can be used by the application", + "p0237": "At any of the above times, the apparatuses transmit a contact establishment signal to the relevancy threshold value setting unit 134 , and the relevancy threshold value setting unit 134 sets the threshold value for the dissimilarity F of related terms used in matching when the contract was established as the initial value of the threshold value for the dissimilarity F of related terms that are to be used when matching processing is started the next time.", + "p0238": "Other Modified Examples of First Embodiment", + "p0239": "Next, other modified examples of the first embodiment will be described.", + "p0240": "FIG. 11 is a block diagram of another modified example of the first embodiment.", + "p0241": "If it is possible to create a database of relevancies between terms and related terms calculated by the relevancy calculation unit 131 , this has an advantage of reducing the number of terms for which relevancy is calculated when matching is performed at a later time.", + "p0242": "In view of this, in another modified example of the first embodiment, a term database 6 of relevancies between terms and related terms calculated by the term relevancy calculation unit 131 is provided. The present embodiment describes an example in which the term database 6 is provided separately from the matching apparatus 1 , and the matching apparatus 1 and the term database 6 are connected via a network, but the matching apparatus 1 may include the term database 6 .", + "p0243": "Also, as shown in FIG. 12 , the matching unit 13 includes a term database updating unit 135 . The term database updating unit 135 has a function by which terms found by the relevancy calculation unit 131 and the related terms thereof are registered in the term database 6 along with a corresponding dissimilarity F calculated by the relevancy calculation unit 131 as one example of relevancy. FIG. 13 is a diagram showing an example of the term database 6 . FIG. 13 shows an example in which related terms for terms are stored in association with corresponding dissimilarity F values.", + "p0244": "According to this configuration, by creating a database of found related terms and calculated relevancy values, it is possible to reduce the amount of processing performed in later matching.", + "p0245": "Second Embodiment", + "p0246": "A second embodiment describes an example in which the relevancy between sensor-side or application-side metadata and a related term is adjusted according to various situations. Note that configurations other than those for adjusting relevancy and extracting sensor candidates are similar to those in the first embodiment, and therefore only differing configurations will be described in detail.", + "p0247": "For example, it is often the case that terms that are added to metadata in parallel are input by the user while recognizing that such terms are related to each other. For example, in the case of a desire for sensing data related to air pressure in order to obtain weather information, there is a high possibility that the user will add \u201cair pressure sensor, weather\u201d to application-side metadata. Similarly, it is often thought to be the case that a user on the sensing data providing side adds terms that are related to each other to metadata.", + "p0248": "In view of this, the second embodiment describes an example in which attention is given to co-occurring terms between sensor-side metadata items and application-side metadata items when the relevancy calculation unit 131 calculates a relevancy. Note that in the second embodiment described below, similarly to the first embodiment, the dissimilarity F is used as an example of the relevancy calculated by the relevancy calculation unit 131 .", + "p0249": "The calculation of the dissimilarity F by the relevancy calculation unit 131 will be described with reference to FIG. 14 .", + "p0250": "FIG. 14 shows an example of the case where \u201cweather\u201d and \u201cpressure sensor\u201d are input at the same time as application-side metadata, that is to say \u201cweather\u201d and \u201cpressure sensor\u201d co-occur as terms in the application-side metadata.", + "p0251": "First, similarly to the first embodiment, the relevancy calculation unit 131 calculates the dissimilarity F between terms and related terms. This is shown specifically below.", + "p0252": "- Dissimilarity F between sensor-side metadata 1 term \u201cabsolute pressure sensor\u201d and related terms dissimilarity F =0.20 dissimilarity F =0.40 - Dissimilarity F between sensor-side metadata 2 term \u201cblood pressure meter\u201d and related terms dissimilarity F =0.50 dissimilarity F =0.10 dissimilarity F =0.10 - Dissimilarity F between sensor-side metadata 3 term \u201cload sensor\u201d and related terms dissimilarity F =0.30 dissimilarity F =0.10 - Dissimilarity F between application-side metadata term \u201cpressure sensor\u201d and related terms dissimilarity F =0.10 dissimilarity F =0.10 dissimilarity F =0.10 dissimilarity F =0.70 dissimilarity F =0.50 dissimilarity F =1.20", + "p0253": "Next, the relevancy calculation unit 131 calculates, as application-side metadata, a co-occurrence C between the term \u201cweather\u201d that co-occurs with \u201cpressure sensor\u201d and related terms. Note that this co-occurrence can also be said to be an indicator of relevancy.", + "p0254": "Here, it is assumed that the term \u201cblood pressure meter\u201d and the term \u201cair pressure sensor\u201d exist as terms that co-occur with the term \u201cweather\u201d, the co-occurrence C is calculated for the term \u201cblood pressure meter\u201d and the term \u201cweather\u201d, and the co-occurrence C is calculated for the term \u201cair pressure sensor\u201d and the term \u201cweather\u201d. The calculation results are as follows.", + "p0255": "co-occurrence C =0.70", + "p0256": "co-occurrence C =0.20", + "p0257": "Based on these results, it can be seen that there is a high possibility that the term \u201cweather\u201d and the term \u201cair pressure sensor\u201d are used together. In view of this, the difference between these co-occurrence C values is reflected in the dissimilarity F between the term \u201cpressure sensor\u201d and the term \u201cblood pressure meter\u201d and the dissimilarity F between the term \u201cpressure sensor\u201d and the term \u201cair pressure sensor\u201d. Specifically, the relevancy between the term \u201cpressure sensor\u201d and the term \u201cblood pressure meter\u201d is lowered , and the relevancy between the term \u201cpressure sensor\u201d and the term \u201cair pressure sensor\u201d is increased . The results of calculating corrected dissimilarity F\u2032 values is shown below.", + "p0258": "dissimilarity F \u2032 +co-occurrence C difference", + "p0259": "=0.50+0.50=1.00", + "p0260": "dissimilarity F \u2032 +co-occurrence C difference", + "p0261": "=0.70\u22120.50=0.20", + "p0262": "These results show that there is a high possibility that the term \u201cpressure sensor\u201d input as application-side metadata is a \u201cpressure sensor\u201d that is related to \u201cweather\u201d, and the related term \u201cair pressure sensor\u201d related to weather has a higher possibility of being related to the term \u201cpressure sensor\u201d input as application-side metadata than the related term \u201cblood pressure meter\u201d does.", + "p0263": "The sensor candidate extraction unit 132 then performs matching using, among the related terms, the related terms that are related to both the sensor-side metadata and the application-side metadata and the dissimilarity F values or dissimilarity F\u2032 values thereof.", + "p0264": "Here, similarly to the first embodiment, in the case where the threshold value for determining the related terms that are to be used is set to 1, only the related term \u201cair pressure sensor\u201d is a related term that is related to both the sensor-side metadata and the application-side metadata and has a dissimilarity F or a dissimilarity F\u2032 less than 1.", + "p0265": "Accordingly, the sensor candidate extraction unit 132 extracts \u201cabsolute pressure sensor\u201d, which is the sensor corresponding to the sensor-side metadata 1 that is related to the related term \u201cair pressure sensor\u201d, as a candidate sensor that can provide sensing data that satisfies the application-side needs.", + "p0266": "Note that the following shows the sum value of the dissimilarity F or the dissimilarity F\u2032 between the common related term and the sensor-side metadata and the dissimilarity F or the dissimilarity F\u2032 between the common related term and the application-side metadata.", + "p0267": "- \u201cabsolute pressure sensor\u201d + dissimilarity F \u2032 \u2212co-occurrence C difference)+dissimilarity F =0.20+0.20 =0.40", + "p0268": "The sensor candidate presentation unit 133 then presents the sensor candidates to the application-side user. FIG. 15 shows an example of presenting sensor candidates to the application-side user. In FIG. 15 , \u201cSensor B\u201d corresponding to \u201cabsolute pressure sensor\u201d in the sensor-side metadata extracted by the sensor candidate extraction unit 132 is presented in a list along with the priority rank and the dissimilarity thereof.", + "p0269": "In comparison with the example shown in FIG. 7 in the first embodiment, only \u201cSensor B\u201d corresponding to \u201cabsolute pressure sensor\u201d is presented to the user, and the dissimilarity value is smaller , and it can be understood that high-precision matching has been performed.", + "p0270": "In this way, rather than merely giving attention to only the terms in respective pieces of metadata, attention is given to the relationship between metadata terms registered by the user, thus realizing higher precision matching that matches the application-side user's desire.", + "p0271": "Note that similarly to the other modified example of the first embodiment, calculated relevancy values may be registered in the term database 6 by the term database updating unit 135 . In this case, co-occurring terms that co-occur with related terms are registered along with related term dissimilarity F\u2032 values that have been corrected according to the relationship with the co-occurring term.", + "p0272": "FIG. 16 is a diagram showing an example of the term database 6 registered or updated by the term database updating unit 135 . In the example in FIG. 16 , each term is stored in association with a related term that is related thereto, the dissimilarity between them, and a co-occurring term that co-occurs with the related term. Note that the related terms for each term also include a related term dissimilarity F\u2032 that has been corrected based on the relationship with the co-occurring term.", + "p0273": "Furthermore, examples of correcting the above-described relevancy between sensor-side or application-side metadata and a related term are not limited to the example of considering the frequency of co-occurrence of a term.", + "p0274": "For example, a sensor-side or application-side metadata term and related term that were used when a contract was established in matching processing can be said to have high relevancy with each other. In view of this, the term database updating unit 135 obtains, from the sensor candidate extraction unit 132 , the number of times that a contract was established in matching processing and the associated sensor-side or application-side metadata terms and related terms, and registers the obtained information in the term database 6 .", + "p0275": "FIG. 17 is a diagram showing an example of the term database 6 registered or updated by the term database updating unit 135 in this case.", + "p0276": "When calculating relevancy, the relevancy calculation unit 131 references the number of times a contract was established in the term database 6 , and performs correction for raising the relevancy between terms the higher the number of contracts were established is. For example, in the example in FIG. 17 , a contract was established one time using the terms \u201cpressure sensor\u201d and \u201cair pressure sensor\u201d, and therefore compared with the dissimilarity F in the case of not giving consideration to the number of times a contract was established as shown in FIG. 16 , the dissimilarity F is corrected from 0.20 to 0.10, thus raising the relevancy. Note that the frequency of contract establishment can also be said to be an indicator of relevancy in the present invention.", + "p0277": "Note that as described above, the following are cases where a contract is established.", + "p0278": "- When a contract for providing sensor sensing data to an application is established by a sensor-side user and an application-side user after matching - When a matching result is received, and then a dataflow control command is transmitted - When a data distribution apparatus having a sensing data distributing function receives a dataflow control command, and sensing data is distributed from a sensor to an application - When a data-using application receives sensing data - When a data-using application has received sensing data, and then it has been checked whether or not the received sensing data can be used by the application", + "p0279": "By calculating relevancy with consideration given to the number of times a contract is established in this way, it is possible to realize higher precision matching that matches the application-side user's desire.", + "p0280": "Third Embodiment", + "p0281": "The first embodiment describes an example in which a related term having a relevancy that is less than a predetermined threshold value is set as a related term that is to be used in matching. A third embodiment describes an example in which, unlike the first embodiment, in the case where there is a related term that is in common between the sensor-side metadata and the application-side metadata, matching is performed using a sum value of the relevancy between the common related term and the sensor-side metadata and the relevancy between the common related term and the application-side metadata. Note that configurations other than that for extracting sensor candidates are similar to those in the first embodiment, and therefore only differing configurations will be described in detail.", + "p0282": "First, the relevancy between metadata terms and related terms calculated by the relevancy calculation unit 131 will be described in detail with reference to FIG. 18 .", + "p0283": "In the following example, it is assumed that the related term a, the related term b, the related term c, and the related term g are found in the thesaurus 2 as related terms that are related to a term X in a sensor-side metadata item.", + "p0284": "The relevancy calculation unit 131 calculates the dissimilarity F between the term X and the related terms that are related to the term X. The dissimilarity F values between the term X and the respective related terms are as shown below.", + "p0285": "dissimilarity F =0.80", + "p0286": "dissimilarity F =0.60", + "p0287": "dissimilarity F =1.20", + "p0288": "dissimilarity F =0.90", + "p0289": "Similarly, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term Y in an application-side metadata item. Here, it is assumed that the related term a, the related term d, the related term e, the related term f, and the related term g are found in the thesaurus 2 as related terms that are related to the term Y in an application-side metadata item.", + "p0290": "The relevancy calculation unit 131 calculates the dissimilarity F between the term Y and the related terms that are related to the term Y. The dissimilarity F values between the term Y and the respective related terms are as shown below.", + "p0291": "dissimilarity F =0.60", + "p0292": "dissimilarity F =0.80", + "p0293": "dissimilarity F =0.60", + "p0294": "dissimilarity F =1.20", + "p0295": "dissimilarity F =1.10", + "p0296": "Next, the sensor candidate extraction unit 132 will be described.", + "p0297": "The sensor candidate extraction unit 132 performs matching between the sensor-side metadata and the application-side metadata based on the found terms and the calculated relevancies obtained by the relevancy calculation unit 131 . The sensor-side metadata and the application-side metadata that are used in matching are sensor-side metadata and application-side metadata in which an acquired related term that is related to a term in the sensor-side metadata and an acquired related term that is related to a term in the application-side metadata are in common with each other. Specifically, they are sensor-side metadata and application-side metadata that both include the same related term with a dissimilarity F value. Matching is performed using the related terms and the dissimilarities F in such sensor-side metadata and application-side metadata.", + "p0298": "Next, the extraction of sensor candidates by the sensor candidate extraction unit 132 will be described in detail with reference to FIG. 18 . In the following description, it is assumed that the term relevancy calculation unit 131 has calculated dissimilarities F as described above.", + "p0299": "In the example described above, the related term a and the related term g are related terms that are in both the sensor-side metadata and the application-side metadata and have a dissimilarity F, that is to say, are related terms that are in common between the sensor-side metadata and the application-side metadata, and the sensor candidate extraction unit 132 performs matching between the sensor-side metadata and the application-side metadata using the related term a and the related term g and the dissimilarity F values for them.", + "p0300": "Specifically, in the case where a related term is in common between the sensor-side metadata and the application-side metadata, a sum value is obtained using the dissimilarity F between the common related term and the sensor-side metadata and the dissimilarity F between the common related term and the application-side metadata.", + "p0301": "The following is the sum value for the dissimilarity F that the related term a has with the sensor-side metadata term X and the application-side metadata term Y.", + "p0302": "dissimilarity F +dissimilarity F", + "p0303": "=0.80+0.60", + "p0304": "=1.40", + "p0305": "The following is the distance between the related term g and the sensor-side metadata term X and application-side metadata term Y.", + "p0306": "dissimilarity F +dissimilarity F", + "p0307": "=0.90+1.10", + "p0308": "=2.00", + "p0309": "Here, as an example of a match determination method, there is a method in which, in the case where there are a plurality of related terms that are in common between the sensor-side metadata and the application-side metadata, it is determined that there is a match between the sensor-side metadata and the application-side metadata if the sum value of the dissimilarity F that each of the related terms has with the sensor-side metadata and the application-side metadata is less than or equal to a predetermined threshold value.", + "p0310": "For example, in the above example, letting the predetermined threshold value be 4, the sum value of the dissimilarity F that the related term a has with the sensor-side metadata term X and the application-side metadata term Y is 1.40, the sum value of the dissimilarity F that the related term g has with the sensor-side metadata term X and the application-side metadata term Y is 2.00, and the sum value thereof is 3.40 . Accordingly, the sum value of all of the dissimilarity F values is less than or equal to the predetermined threshold value 4, and therefore it is determined that there is a match between the sensor-side metadata and the application-side metadata.", + "p0311": "As another example of a match determination method, there is a method in which, in the case where there are a plurality of related terms that are in common between the sensor-side metadata and the application-side metadata, the smallest dissimilarity F with the sensor-side metadata and the application-side metadata among all of the related terms is selected, and it is determined that there is a match between the sensor-side metadata and the application-side metadata if the selected dissimilarity F is less than or equal to the predetermined threshold value. The sensor corresponding to that sensor-side metadata is then extracted as a candidate sensor that can provide sensing data that satisfies the application's needs.", + "p0312": "For example, in the above example, the value of the dissimilarity F that the related term a has with the sensor-side metadata term X and the application-side metadata term Y is 1.40, the value of the dissimilarity F that the related term g has with the sensor-side metadata term X and the application-side metadata term Y is 2.00, and therefore the sensor-side metadata and application-side metadata having the related term a in common are selected. Here, in the case where the predetermined threshold value is 1.50, if the dissimilarity F is less than or equal to 1.50, it is determined that there is a match between the sensor-side metadata and the application-side metadata. In this case, in the above example, the value of the dissimilarity F that the related term a has with the sensor-side metadata and the application-side metadata is 1.40, and therefore it is determined that there is a match between the sensor-side metadata and the application-side metadata that have the related term a in common. The sensor corresponding to that sensor-side metadata is then extracted as a candidate sensor that can provide sensing data that satisfies the application's needs.", + "p0313": "Note that the match determination method is not limited to the method described above. For example, there is a method in which the relevancy that a related term has with sensor-side metadata and application-side metadata can be calculated without providing a threshold value as described above, that is to say, a configuration is possible in which all sensor-side metadata and application-side metadata that have a related term in common are set as matching candidates, and the sensors corresponding to such sensor-side metadata are extracted as candidates sensors that can provide sensing data that satisfies an application's needs.", + "p0314": "Next, a specific example of operations of the relevancy calculation unit 131 , the sensor candidate extraction unit 132 , and the sensor candidate presentation unit 133 will be described with reference to FIG. 19 .", + "p0315": "In the following description, the sensors include an \u201cabsolute pressure sensor\u201d, a \u201cblood pressure meter\u201d, and a \u201cload sensor\u201d, and the sensor-side metadata includes the term \u201cabsolute pressure sensor\u201d in sensor-side metadata 1 for the \u201cabsolute pressure sensor\u201d, the term \u201cblood pressure meter\u201d in sensor-side metadata 2 for the \u201cblood pressure meter\u201d, and the term \u201cload sensor\u201d in sensor-side metadata 3 for the \u201cload sensor\u201d. Also, the term \u201cpressure sensor\u201d is included as application-side metadata.", + "p0316": "First, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 . Here, the related term \u201cair pressure sensor\u201d and the related term \u201ctemperature sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 . The dissimilarity F is then calculated between the term \u201cabsolute pressure sensor\u201d in the sensor-side metadata 1 and each of the related terms. The calculation results are as follows.", + "p0317": "dissimilarity F =0.20", + "p0318": "dissimilarity F =0.40", + "p0319": "Next, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 . Here, the related term \u201cpressure sensor\u201d, the related term \u201cblood pressure monitor\u201d, and the related term \u201cblood pressure sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 . The dissimilarity F is then calculated between the term \u201cblood pressure meter\u201d in the sensor-side metadata 2 and each of the related terms. The calculation results are as follows.", + "p0320": "dissimilarity F =0.50", + "p0321": "dissimilarity F =0.10", + "p0322": "dissimilarity F =0.10", + "p0323": "Furthermore, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cload sensor\u201d in the sensor-side metadata 3 . Here, the related term \u201cstrain gauge\u201d and the related term \u201cweight sensor\u201d are found in the thesaurus 2 as related terms for the term \u201cload sensor\u201d in the sensor-side metadata 3 . The dissimilarity F is then calculated between the term \u201cload sensor\u201d in the sensor-side metadata 3 and each of the related terms. The calculation results are as follows.", + "p0324": "dissimilarity F =0.30", + "p0325": "dissimilarity F =0.10", + "p0326": "Furthermore, the relevancy calculation unit 131 searches the thesaurus 2 for related terms that are related to the term \u201cpressure sensor\u201d in the application-side metadata. Here, the related term \u201cpressure meter\u201d, the related term \u201cpressure gauge\u201d, the related term \u201cpressure monitor\u201d, the related term \u201cair pressure sensor\u201d, the related term \u201cblood pressure meter\u201d, and the related term \u201cstrain gauge\u201d are found in the thesaurus 2 as related terms for the term \u201cpressure sensor\u201d in the application-side metadata. The dissimilarity F is then calculated between the term \u201cpressure sensor\u201d in the application-side metadata and each of the related terms. The calculation results are as follows.", + "p0327": "dissimilarity F =0.10", + "p0328": "dissimilarity F =0.10", + "p0329": "dissimilarity F =0.10", + "p0330": "dissimilarity F =0.70", + "p0331": "dissimilarity F =0.50", + "p0332": "dissimilarity F =1.20", + "p0333": "The sensor candidate extraction unit 132 then performs matching using, among the related terms, the related terms that are related to both the sensor-side metadata and the application-side metadata and the dissimilarity F values thereof. Specifically, the application-side metadata 1 and the sensor-side metadata 1 are related via the related term \u201cair pressure sensor\u201d , the application-side metadata 1 and the sensor-side metadata 2 are related via the related terms \u201cblood pressure meter\u201d and \u201cpressure sensor\u201d , and the application-side metadata 3 and the sensor-side metadata 2 are related via the related term \u201cstrain gauge\u201d . Also, the sum values of the dissimilarity F that the related terms have with the sensor-side metadata and the application-side metadata are as shown below.", + "p0334": "- sum value of dissimilarity F for related term \u201cair pressure sensor\u201d dissimilarity F +dissimilarity F =0.70+0.20 =0.90 - sum value of dissimilarity F for related term \u201cblood pressure meter\u201d dissimilarity F +dissimilarity F =0.50+0.0 =0.5 - sum value of dissimilarity F for related term \u201cstrain gauge\u201d dissimilarity F +dissimilarity F =1.20+0.30 =1.50", + "p0335": "Next, upon arranging the sensor-side metadata terms in order of lowest sum value of dissimilarity F, the order is \u201cblood pressure meter\u201d, \u201cabsolute pressure sensor\u201d, and \u201cload sensor\u201d. In this example, the sensor candidate extraction unit 132 extracts all of the matching sensors, that is to say \u201cblood pressure meter\u201d, \u201cabsolute pressure sensor\u201d, and \u201cload sensor\u201d, as candidate sensors that can provide sensing data that satisfies an application's needs.", + "p0336": "At this time, a priority ranking may be applied to the extracted \u201cblood pressure meter\u201d, \u201cabsolute pressure sensor\u201d, and \u201cload sensor\u201d. Then, this content may be presented to the application-side user by the sensor candidate presentation unit 133 . FIG. 20 shows an example of presenting sensor candidates to the application-side user. In FIG. 20 , \u201cSensor A\u201d corresponding to \u201cblood pressure meter\u201d in the sensor-side metadata, \u201cSensor B\u201d corresponding to \u201cabsolute pressure sensor\u201d in the sensor-side metadata, and \u201cSensor C\u201d corresponding to \u201cload sensor\u201d in the sensor-side metadata, which were extracted by the sensor candidate extraction unit 132 , are presented in a list along with the priority ranks and the sum values of the dissimilarity F thereof. The application-side user can view the presented list of sensor candidates and selected a desired sensor.", + "p0337": "Operations in First to Third Embodiments", + "p0338": "Next, operations in the above first to third embodiments will be described with reference to the flowcharts in FIGS. 21 and 22 .", + "p0339": "The sensor-side metadata acquisition unit 11 acquires sensor-side metadata .", + "p0340": "The relevancy calculation unit 131 searches the thesaurus 2 or the term database 6 for related terms that are related to the terms in the sensor-side metadata items .", + "p0341": "The relevancy calculation unit 131 calculates a dissimilarity F between the sensor-side metadata and the related terms . Note that the relevancy calculation unit 131 does not need to calculate the dissimilarity F if the dissimilarity F can be obtained from the term database 6 .", + "p0342": "The application-side metadata acquisition unit 12 acquires application-side metadata .", + "p0343": "The relevancy calculation unit 131 searches the thesaurus 2 or the term database 6 for related terms that are related to terms in the application-side metadata items .", + "p0344": "The relevancy calculation unit 131 calculates a dissimilarity F between the application-side metadata and the related terms . Note that the relevancy calculation unit 131 does not need to calculate the dissimilarity F if the dissimilarity F can be obtained from the term database 6 .", + "p0345": "The sensor candidate extraction unit 132 performs processing for matching between the sensor-side metadata and the application-side metadata based on the found related terms and the calculated dissimilarity F values thereof obtained by the relevancy calculation unit 131 . Specific processing for matching has been described above.", + "p0346": "If there is a match between sensor-side metadata and application-side metadata , the dataflow control command instruction unit 14 outputs a dataflow control command . Note that the following cases are examples of triggers for the output of the dataflow control command by the dataflow control command instruction unit 14 , as described above.", + "p0347": "Case where the matching unit 13 determines, based on a predetermined condition, that there is a match between sensor-side metadata and application-side metadata", + "p0348": "Case where the sensor candidate presentation unit 133 of the matching unit 13 presents, to the application-side user, a matching result between sensor-side metadata and application-side metadata obtained, and the application-side user desires, based on the matching result, the provision of sensing data", + "p0349": "The above cases are merely examples, and there is no limitation to the above cases.", + "p0350": "However, if there is no match between the sensor-side metadata and the application-side metadata , match-not-found processing is performed .", + "p0351": "The following describes match-not-found processing.", + "p0352": "In match-not-found processing, it is determined whether or not there is other processing that can be applied . If any of the following processing can be applied, such processing is performed.", + "p0353": "- First modified example and second modified example of first embodiment: change threshold value of dissimilarity F used when selecting related terms that are to be used in matching - Second embodiment: correct dissimilarity F of related term with consideration given to co-occurrence of term in sensor-side metadata or application-side metadata, number of times a contract was established, or the like - Third embodiment: change threshold value of sum value of dissimilarity F for determining match between sensor-side metadata and application-side metadata .", + "p0354": "At least one of the above processing is performed, and then processing for matching between sensor-side metadata and application-side metadata is performed again .", + "p0355": "However, if there is no processing that can be applied, match-not-found processing is ended.", + "p0356": "Note that although the example of performing the second embodiment is described as other processing that can be applied in match-not-found processing, it goes without saying that the related term dissimilarity F may be corrected in the first matching processing step .", + "p0357": "Also, the relevancy is not limited to being the dissimilarity F, and need only be an indicator of relevancy between terms.", + "p0358": "This concludes the description of operations in the first to third embodiments.", + "p0359": "Note that as shown in FIG. 23 , the matching apparatus 1 can be configured by a general-purpose computer that has hardware resources such as a processor 201 , a memory 202 , a storage device 203 , an input device 204 , a display device 205 , and a communication apparatus 206 . The functions of the matching apparatus 1 are realized by a program stored in the storage device 203 being loaded to the memory 202 and executed by the processor 201 . Note that the matching apparatus 1 may be configured by one computer, or may be configured by distributed computing with use of multiple computers. Also, in order to increase the processing speed, some or all of the functions of the matching apparatus 1 can also be realized using dedicated hardware .", + "p0360": "Also, part or all of the above embodiments can be described as noted below, but there is no limitation to this.", + "p0361": "Note 1", + "p0362": "A matching apparatus having at least one or more hardware processor,", + "p0363": "the processor", + "p0364": "- acquiring sensor-side metadata that is information regarding a sensor that outputs sensing data; - acquiring application-side metadata that is information regarding an application that provides a service with use of the sensing data; - performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; and - transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0365": "Note 2", + "p0366": "A matching method in which a computer", + "p0367": "acquires sensor-side metadata that is information regarding a sensor that outputs sensing data;", + "p0368": "acquires application-side metadata that is information regarding an application that provides a service with use of the sensing data;", + "p0369": "performs matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracts a candidate sensor that can provide sensing data that matches needs of the application; and", + "p0370": "transmits, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0371": "Note 3", + "p0372": "A recording medium having stored thereon a program that causes a computer to execute:", + "p0373": "a process of acquiring sensor-side metadata that is information regarding a sensor that outputs sensing data;", + "p0374": "a process of acquiring application-side metadata that is information regarding an application that provides a service with use of the sensing data;", + "p0375": "a process of performing matching between the sensor-side metadata and the application-side metadata in which a first related term that is related to a term in the acquired sensor-side metadata and a second related term that is related to a term in the acquired application-side metadata are in common with each, and extracting a candidate sensor that can provide sensing data that matches needs of the application; and", + "p0376": "a process of transmitting, to a sensor management apparatus that manages the sensor, a dataflow control command that includes information that specifies the extracted sensor and the application.", + "p0377": "Although the present invention has been described by way of preferred embodiments and modified examples above, it is not necessary to include the configurations of all of the embodiments or modified examples, and not only is implementation by appropriate combinations possible, but also the present invention is not necessarily limited to the above embodiments and modified examples, and various modifications can be made within the scope of the technical idea.", + "p0378": "INDEX TO THE REFERENCE NUMERALS", + "p0379": "- 1 matching apparatus - 2 thesaurus - 3 sensor data providing system - 4 data distribution control apparatus - 5 application system - 6 term database - 11 sensor-side metadata acquisition unit - 12 application-side metadata acquisition unit - 13 matching unit - 14 dataflow control command instruction unit - 131 relevancy calculation unit - 132 sensor candidate extraction unit - 133 sensor candidate presentation unit - 134 relevancy threshold value setting unit - 135 term database updating unit - 201 processor - 202 memory - 203 storage device - 204 input device - 205 display device - 206 communication apparatus", + "features": { + "F1": "A processor is configured with a program to function as a sensor-side metadata acquisition unit that acquires information regarding a sensor that outputs sensing data.", + "F2": "The processor functions as an application-side metadata acquisition unit that acquires information regarding an application providing a service using the sensing data.", + "F3": "The processor functions as a matching unit performing matching between sensor-side metadata and application-side metadata, identifying a candidate sensor that provides matching data required by the application.", + "F4": "The matching unit includes a relevancy calculation unit that calculates a relevancy between terms in the sensor-side metadata and related terms in the application-side metadata.", + "F5": "The processor includes a sensor candidate extraction unit that, using relevancy calculations, matches metadata to extract a candidate sensor meeting application needs.", + "F6": "The processor functions as an instruction unit that transmits a dataflow control command to a sensor management apparatus, specifying the sensor and application.", + "F7": "The processor performs matching with the sensor-side metadata and application-side metadata using related terms having relevancy within a predetermined range.", + "F8": "A range setting unit in the processor sets the predetermined range of relevancy for terms to be used in matching.", + "F9": "In cases of established contracts through matching, the range setting unit sets the relevancy range based on previously established conditions.", + "F10": "The sensor candidate extraction unit corrects relevancy based on co-occurrence with terms in metadata for accurate matching.", + "F11": "The sensor candidate extraction unit corrects relevancy according to the frequency of contract establishment with previously used terms.", + "F12": "The processor includes a term database storing terms from metadata, related terms, and relevancy associations, which is updated with new matching data.", + "F13": "A sensor candidate presentation unit presents extracted candidate sensor information to the application-side user.", + "F14": "A matching method includes acquiring sensor-side metadata and application-side metadata, performing matching, and calculating relevancy to extract candidate sensors.", + "F15": "A non-transitory computer-readable storage medium stores a program causing a computer to perform processes of metadata acquisition, matching, and sensor extraction." + } + } + }, + { + "FAN": "100251983", + "CPC": [ + "G06F-016/00", + "G06F-016/3329", + "G06F-016/3344", + "G06F-040/30" + ], + "Content": { + "title": "Method and apparatus for determining information", + "pa01": "The present application relates to the technical field of computers. Disclosed are a method and apparatus for determining information. The method comprises: acquiring overall preceding text information and a plurality of pieces of candidate reply information for replying to the overall preceding text information; for each of the plurality of pieces of candidate reply information, determining total difference information between the candidate reply information and candidate reply information of the plurality of pieces of candidate reply information other than the candidate reply information; determining consistency information between the total difference information and the overall preceding text information as first consistency information between the candidate reply information and the overall preceding text information; and according to the first consistency information corresponding to each piece of candidate reply information, determining target reply information from among the plurality of pieces of candidate reply information.", + "c-en-0001": "A method for determining information, the method comprising: acquiring whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information; determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information; determining consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information; and determining target reply information from the plurality pieces of candidate reply information, based on first consistency information corresponding to each piece of candidate reply information.", + "c-en-0002": "The method according to claim 1, wherein the determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, the total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information, comprises: acquiring word vectors of each piece of candidate reply information; for each piece of candidate reply information, determining, based on a similarity between word vectors of the piece of candidate reply information and word vectors of each piece of the other candidate reply information, difference information between the piece of candidate reply information and each piece of the other candidate reply information; and determining difference information between the piece of candidate reply information and all of the other candidate reply information as the total difference information between the piece of candidate reply information and the other candidate reply information.", + "c-en-0003": "The method according to claim 2, wherein the determining the consistency information between the total difference information and the whole preceding text information as the first consistency information between the piece of candidate reply information and the whole preceding text information, comprises: acquiring word vectors of the whole preceding text information; and determining consistency information between the total difference information and the word vectors of the whole preceding text information as the first consistency information between the piece of candidate reply information and the whole preceding text information.", + "c-en-0004": "The method according to claim 1, wherein the method further comprises: determining consistency information between the total difference information and target preceding text information as second consistency information between the piece of candidate reply information and the target preceding text information, wherein the target preceding text information is information in the whole preceding text information that belongs to a same interlocutor as the piece of candidate reply information; and the determining the target reply information from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information, comprises: determining the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information.", + "c-en-0005": "The method according to claim 4, wherein the total difference information is determined based on a similarity between word vectors of the piece of candidate reply information and word vectors of the other candidate reply information of the plurality pieces of candidate reply information other than the piece of candidate reply information; the determining the consistency information between the total difference information and the target preceding text information as the second consistency information between the piece of candidate reply information and the target preceding text information, comprises: acquiring word vectors of the target preceding text information; and determining consistency information between the total difference information and the word vectors of the target preceding text information as the second consistency information between the piece of candidate reply information and the target preceding text information.", + "c-en-0006": "The method according to claim 4, wherein the determining the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information, comprises: for each piece of candidate reply information, determining a score of the piece of candidate reply information, on basis of the first consistency information corresponding to the piece of candidate reply information, the second consistency information corresponding to the piece of candidate reply information, and third consistency information between semantics of the piece of candidate reply information and semantics of the whole preceding text information; and determining the target reply information from the plurality pieces of candidate reply information, based on a score of each piece of candidate reply information.", + "c-en-0007": "An apparatus for determining information, the apparatus comprising: an acquisition unit, configured to acquire whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information; a first determining unit, configured to determine, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information; a second determining unit, configured to determine consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information; and a third determining unit, configured to determine target reply information from the plurality pieces of candidate reply information, based on first consistency information corresponding to each piece of candidate reply information.", + "c-en-0008": "The apparatus according to claim 7, wherein the first determining unit comprises: a first acquisition module, configured to acquire word vectors of each piece of candidate reply information; a first determining module, configured to determine, for each piece of candidate reply information, based on a similarity between word vectors of the piece of candidate reply information and word vectors of each piece of the other candidate reply information, difference information between the piece of candidate reply information and each piece of the other candidate reply information; and a second determining module, configured to determine difference information between the piece of candidate reply information and all of the other candidate reply information as the total difference information between the piece of candidate reply information and the other candidate reply information.", + "c-en-0009": "The apparatus according to claim 7, wherein the second determining unit comprises: a second acquisition module, configured to acquire word vectors of the whole preceding text information; and a third determining module, configured to determine consistency information between the total difference information and the word vectors of the whole preceding text information as the first consistency information between the piece of candidate reply information and the whole preceding text information.", + "c-en-0010": "The apparatus according to claim 7, wherein the apparatus further comprises: a fourth determining unit, configured to determine consistency information between the total difference information and target preceding text information as second consistency information between the piece of candidate reply information and the target preceding text information, wherein the target preceding text information is information in the whole preceding text information that belongs to a same interlocutor as the piece of candidate reply information; and the third determining unit comprises: a fourth determining module, configured to determine the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information.", + "c-en-0011": "The apparatus according to claim 10, wherein the total difference information is determined based on a similarity between word vectors of the piece of candidate reply information and word vectors of the other candidate reply information of the plurality pieces of candidate reply information other than the piece of candidate reply information, and the fourth determining unit comprises: a third acquisition module, configured to acquire word vectors of the target preceding text information; and a fifth determining module, configured to determine consistency information between the total difference information and the word vectors of the target preceding text information as the second consistency information between the piece of candidate reply information and the target preceding text information.", + "c-en-0012": "The apparatus according to claim 10, wherein the fourth determining module comprises: a scoring module, configured to determine, for each piece of candidate reply information, a score of the piece of candidate reply information, on basis of the first consistency information corresponding to the piece of candidate reply information, the second consistency information corresponding to the piece of candidate reply information, and third consistency information between semantics of the piece of candidate reply information and semantics of the whole preceding text information; and a selection module, configured to determine the target reply information from the plurality pieces of candidate reply information, based on a score of each piece of candidate reply information.", + "c-en-0013": "An electronic device, comprising: at least one processor; and a memory, in communication with the at least one processor; wherein, the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-6.", + "c-en-0014": "A non-transitory computer readable storage medium, storing a computer instruction thereon, wherein the computer instruction is used to cause the computer to perform the method according to any one of claims 1-6.", + "c-en-0015": "A computer program product, comprising a computer program, the computer program, when executed by a processor, implements the method according to any one of claims 1-6.", + "p0001": "This application claims the priority and rights to Chinese Patent Application No. 202210151466.6 filed with the China National Intellectual Property Administration on February 17, 2022 and entitled \"method and apparatus for determining information\", the contents of which are hereby incorporated by reference in their entirety.", + "p0002": "TECHNICAL FIELD", + "p0003": "The present disclosure relates to the field of computer technology, and in particular, to a method and apparatus for determining information.", + "p0004": "BACKGROUND", + "p0005": "With the development of AI technology, AI models have been adopted in more and more fields to implement human-machine dialogue systems, such as intelligent customer service systems, intelligent chat systems, and self-service question-and-answer systems. Existing human-machine dialogue systems usually determine reply/return information based on semantics of whole preceding information in historical dialogues.", + "p0006": "However, a method for determining reply/return information based on semantics of overall preceding information suffers from inaccurate determination of the information.", + "p0007": "SUMMARY", + "p0008": "Embodiments of the present disclosure provides a method and apparatus for determining information, an electronic device, a computer readable storage medium, and a computer program product.", + "p0009": "Some embodiments of the present disclosure provide a method for determining information, including: acquiring whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information; determining, for each of the plurality pieces of candidate reply information, total difference information between the candidate reply information and other candidate reply information of the plurality pieces of candidate reply information other than the candidate reply information; determining consistency information between the total difference information and the whole preceding text information as first consistency information between the candidate reply information and the whole preceding text information; and determining target reply information from among the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information.", + "p0010": "Some embodiments of the present disclosure provide an apparatus for determining information, including: an acquisition unit, configured to acquire whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information; a first determining unit, configured to determine, for each of the plurality pieces of candidate reply information, total difference information between the candidate reply information and other candidate reply information of the plurality pieces of candidate reply information other than the candidate reply information; a second determining unit, configured to determine consistency information between the total difference information and the whole preceding text information as first consistency information between the candidate reply information and the whole preceding text information; and a third determining unit, configured to determine target reply information from among the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information.", + "p0011": "Some embodiments of the present disclosure provide an electronic device, including: one or more processors; a storage apparatus, for storing one or more programs, the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for determining information as provided in the first aspect.", + "p0012": "Some embodiments of the present disclosure provide a computer readable storage medium, storing a computer program thereon, the program, when executed by a processor, implements any embodiment of the method for determining information as described above.", + "p0013": "Some embodiments of the present disclosure provide a computer program product, including a computer program, the program, when executed by a processor, implements any embodiment of the method for determining information as described above.", + "p0014": "It should be understood that the content described in this section is not intended to identify critical or important features of embodiments of the present disclosure, and is not used to limit the scope of the present disclosure. Other features of the present disclosure will become readily comprehensible through the following description.", + "p0015": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0016": "The accompanying drawings are used for better understanding of this solution, and do not constitute a limitation to the present disclosure. In which:", + "p0017": "FIG. 1 is an example system architecture in which an embodiment of the present disclosure may be applied; FIG. 2 is a flowchart of a method for determining information according to an embodiment of the present disclosure; FIG. 3 is a flowchart of a method for determining information according to another embodiment of the present disclosure; FIG. 4 is a flowchart of an application scenario of the method for determining information according to an embodiment of the present disclosure; FIG. 5 is a schematic diagram of a training apparatus for a model used in an application scenario in the method for determining information according to an embodiment of the present disclosure; FIG. 5 is a schematic structural diagram of an apparatus for determining information according to an embodiment of the present disclosure; and FIG. 7 is a block diagram of an electronic device used to implement the method for determining information of embodiments of the present disclosure.", + "p0018": "DETAILED DESCRIPTION OF EMBODIMENTS", + "p0019": "Exemplary embodiments of the present disclosure are described below in combination with the accompanying drawings, and various details of embodiments of the present disclosure are included in the description to facilitate understanding, and should be considered as exemplary only. Accordingly, it should be recognized by one of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for clarity and conciseness, descriptions for well-known functions and structures are omitted in the following description.", + "p0020": "FIG. 1 illustrates an example system architecture 100 in which a method for determining information or an apparatus for determining information according to an embodiment of the present disclosure may be applied.", + "p0021": "As shown in FIG. 1 , the system architecture 100 may include terminal device 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium providing a communication link between the terminal device 101, 102, 103 and the server 105. The network 104 may include various types of connections, for example, wired or wireless communication links, or optical fiber cables.", + "p0022": "A user may use the terminal device 101, 102, 103 to interact with the server 105 via the network 104, to receive or send messages, etc. The terminal device 101, 102, 103 may be user terminal devices on which various client applications may be installed, such as an image-based application, a video-based application, a shopping-based application, a chat-based application, a search-based application, or a financial application.", + "p0023": "The terminal device 101, 102, 103 may be various electronic devices having a display screen and supporting receiving messages from the server, including, but not limited to, a smartphone, a tablet, an e-book reader, an e-player, a laptop and a desktop computer.", + "p0024": "The terminal device 101, 102, 103 may be hardware or software. When being the hardware, the terminal device 101, 102, 103 may be various electronic devices, when being the software, the terminal device 101, 102, 103 may be installed in the above electronic devices. The terminal devices may be implemented as a plurality pieces of software or a plurality of software modules , or may be implemented as a single piece of software or a single software module, which will not be limited herein.", + "p0025": "The server 105 may acquire whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information, then determine, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and the plurality pieces of candidate reply information, determine consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information, and determine target reply information from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information.", + "p0026": "It should be noted that the method for determining information provided in embodiments of the present disclosure may be performed by the server 105, correspondingly, the apparatus for determining information may be provided in the server 105.", + "p0027": "It should be appreciated that the numbers of the terminal devices, the networks and the servers in FIG. 1 are merely illustrative. Any number of terminal devices, networks and servers may be provided depending on implementation needs.", + "p0028": "With further reference to FIG. 2 , a flowchart 200 of a method for determining information according to an embodiment of the present disclosure is shown, including the following steps:", + "p0029": "Step 201, acquiring whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information.", + "p0030": "In the present embodiment, an executing body of the method for determining information may acquire the whole preceding text information and the plurality pieces of candidate reply information for replying to the whole preceding text information. Here, the whole preceding text information refers to historical information in a dialogue, such as historical question-and-answer messages in an intelligent customer service system, or a historical dialogue record in an intelligent chat system. The reply information refers to reply information used to reply to the last piece of information, or all pieces of information, in the whole preceding text information.", + "p0031": "Step 202, determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information.", + "p0032": "In the present embodiment, for each piece of candidate reply information in the plurality pieces of candidate reply information, the total difference information between the piece of candidate reply information and the other candidate reply information of the plurality pieces of candidate reply information other than the candidate reply information may be determined. For example, if candidate reply information A is \"It is a sunny day today\", candidate reply information B is \"It is a sunny day with moderate humidity\", candidate reply information C is \"It is a sunny day, suitable for travelling\", candidate reply information D is \"It is a sunny day, suitable for travelling, suitable for hiking\", then the total difference information between the candidate reply information D and the rest candidate reply information in the plurality pieces of candidate reply information other than the candidate reply information is \"suitable for travelling, suitable for hiking\".", + "p0033": "Step 203, determining consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information.", + "p0034": "In the present embodiment, consistency information between the piece of candidate reply information and the whole preceding text information may be determined based on the total difference information between the piece of candidate reply information and the other candidate reply information other than the candidate reply information. For ease of differentiation, this consistency information may be referred to as the first consistency information, which is used to represent the degree of logical consistency/degree of similarity between the content expressed in the total difference information and the content expressed in the whole preceding text information, or used to represent the degree of logical consistency/degree of similarity between the syntax/semantics of the total difference information and the syntax/semantics of the whole preceding text information, etc.", + "p0035": "Step 204, determining target reply information from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information.", + "p0036": "In the present embodiment, the target reply information may be determined from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information. Here, the first consistency information corresponding to any one piece of reply information refers to: the consistency information between the total difference information and the whole preceding text information.", + "p0037": "The present embodiment provides a method for determining information, by acquiring whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information; determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the candidate reply information; determining consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information; and determining target reply information from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information, the method may enhance the logical consistency and content consistency of the determined target reply information with the whole preceding text information in the historical dialogue record, thereby improving an accuracy of the pushed information.", + "p0038": "Alternatively, the determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the candidate reply information, includes: acquiring word vectors of each piece of candidate reply information; for each piece of candidate reply information, determining, based on a similarity between the word vectors of the candidate reply information and the word vectors of each piece of the other candidate reply information, difference information between the piece of candidate reply information and each piece of other candidate reply information; and determining the difference information between the piece of candidate reply information and all of the other candidate reply information as the total difference information between the piece of candidate reply information and the other candidate reply information.", + "p0039": "In the present embodiment, the word vectors of each piece of candidate reply information may be acquired based on a pre-trained semantic parsing model, or a pre-trained linguistic representation model, or the like, so as to determine, for each piece of candidate reply information, based on the similarity between the word vectors of the piece of candidate reply information and the word vectors of each piece of the other candidate reply information in the plurality pieces of candidate reply information, the difference information between the piece of candidate reply information and each piece of candidate reply information. Here, the difference information may be difference content between the semantics expressed by the two pieces of information, or difference information between the logics expressed by the two pieces of information. Then, the difference information between the piece of candidate reply information and each piece of the other candidate reply information may be aggregated/gathered to obtain the total difference information between the piece of candidate reply information and all pieces of the other candidate reply information, i.e., the total difference information between the piece of candidate reply information and the other candidate reply information in the plurality pieces of candidate reply information other than the candidate reply information.", + "p0040": "In the present embodiment, the difference information between pieces of candidate reply information is determined based on the similarity between the word vectors of the pieces of candidate reply information, which may improve an accuracy as well as an efficiency of determining the difference information.", + "p0041": "Alternatively, the determining consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information, includes: acquiring word vectors of the whole preceding text information; and determining consistency information between the total difference information and the word vectors of the whole preceding text information as the first consistency information between the piece of candidate reply information and the whole preceding text information.", + "p0042": "In the present embodiment, the word vectors of the whole preceding text information may be acquired based on a pre-trained semantic parsing model, or a pre-trained linguistic representation model, or the like, then the first consistency information between the piece of candidate reply information and the whole preceding text information may be determined based on the word vectors of the whole preceding text information as well as the total difference information between the piece of candidate reply information and the other candidate reply information. It may be understood that since the total difference information is determined based on differences between the word vectors of the piece pf candidate reply information, a data format of the total difference information is in the form of word vector, and based on the word vectors of the whole preceding text information and the total difference information, a consistency comparison, i.e., similarity/homogeneity comparison, may be directly performed to obtain the consistency information between the piece of candidate information and the whole preceding text information.", + "p0043": "In the present embodiment, the first consistency information between the piece of candidate reply information and the whole preceding text information is determined based on the word vectors of the whole preceding text information and the total difference information, which may improve an accuracy as well as an efficiency of determining the first consistency information.", + "p0044": "With further reference to FIG. 3 , a flowchart 300 of the method for determining information according to another embodiment of the present disclosure is shown, including the following steps.", + "p0045": "Step 301, acquiring whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information.", + "p0046": "Step 302, determining, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information.", + "p0047": "Step 303, determining consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information.", + "p0048": "In the present embodiment, description of step 301, step 302, and step 303 is consistent with the description of step 201, step 202, and step 203, and detailed description thereof will be omitted.", + "p0049": "Step 304, determining consistency information between the total difference information and target preceding text information as second consistency information between the piece of candidate reply information and the target preceding text information, where, the target preceding text information is information in the whole preceding text information that belongs to a same interlocutor as the piece of candidate reply information.", + "p0050": "In the present embodiment, the consistency information between the total difference information and the target preceding text information may be determined as consistency information between the piece of candidate reply information and the target preceding text information, which may be referred to as the second consistency information, for ease of differentiation. This consistency information is used to represent the logical consistency between the content expressed in the total difference information and the content expressed in the target whole preceding text information, or used to represent the logical consistency between the syntax/semantics of the total difference information and the syntax/semantics of the target whole preceding text information, etc. The target preceding text information refers to: a dialogue record generated by the interlocutor of the piece of candidate reply information in the whole preceding text information. For example, the whole preceding text information is a dialogue between two parties A and B and B may be a robot customer service for replying to the user's question ), and if the current dialogue process is at a stage where B needs to reply, i.e., currently, a message needs to be generated by B, then the interlocutor of the piece of candidate reply information is B. Therefore, the target preceding text information is all information from B in the whole preceding text information.", + "p0051": "Step 305, determining the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information.", + "p0052": "In the present embodiment, the target reply information may be determined from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information. Here, the first consistency information corresponding to any one piece of reply information refers to: the consistency information between the total difference information and the whole preceding text information; and the second consistency information corresponding to any one piece of reply information refers to: the consistency information between the total difference information and the target preceding text information.", + "p0053": "The method for determining information provided in embodiments of the present embodiment, compared to the method described in the embodiment of FIG. 2 , adds the step of acquiring second consistency information between the piece of candidate reply information and the target preceding text information, as well as determining the target reply information also basing on the second consistency information, may enhance the logical consistency and content consistency between the determined candidate reply information and the target preceding text information in the historical dialogue record, that is, enhance the consistency of information provided by the interlocutor of the candidate reply information, thereby improving an accuracy of determining the reply information.", + "p0054": "Alternatively, the total difference information is determined based on the similarity between the word vectors of the piece of candidate reply information and the word vectors of the other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information, the determining consistency information between the total difference information and target preceding text information as second consistency information between the piece of candidate reply information and the target preceding text information, includes: acquiring word vectors of the target preceding text information; and determining consistency information between the total difference information and the word vectors of the target preceding text information as the second consistency information between the piece of candidate reply information and the target preceding text information.", + "p0055": "In the present embodiment, the word vectors of the target preceding text information may be acquired based on a pre-trained semantic parsing model, or a pre-trained linguistic representation model, or the like, then the second consistency information between the piece of candidate reply information and the target preceding text information may be determined based on the word vectors of the target preceding text information as well as the total difference information between the piece of candidate reply information and the other candidate reply information. It may be understood that since the total difference information is determined based on differences between the word vectors of the candidate reply information and the word vectors of the other candidate reply information, a data format of the total difference information is in the form of word vector. Consistency comparison, i.e., similarity/homogeneity comparison, may be directly performed on the word vectors of the target preceding text information and the total difference information, to obtain the consistency information between the piece of candidate information and the target preceding text information.", + "p0056": "In the present embodiment, the second consistency information between the piece of candidate reply information and the whole preceding text information is determined based on the word vectors of the target preceding text information and the total difference information, which may improve an accuracy as well as an efficiency of determining the second consistency information.", + "p0057": "Alternatively, the determining the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information, includes: for each piece of candidate reply information, determining a score of the piece of candidate reply information, on basis of the first consistency information corresponding to the piece of candidate reply information, the second consistency information corresponding to the piece of candidate reply information, and third consistency information between semantics of the piece of candidate reply information and semantics of the whole preceding text information; and determining the target reply information from the plurality pieces of candidate reply information, based on score of each piece of candidate reply information.", + "p0058": "In the present embodiment, for each piece of candidate reply information, the first consistency information corresponding to the piece of candidate reply information, the second consistency information corresponding to the piece of candidate reply information, and the third consistency information between the semantics of the piece of candidate reply information and the semantics of the whole preceding text information may be used, to determine the score of the piece of candidate reply information. In particular, the score of the piece of candidate reply information is made positively proportional to a preceding text semantic relevance , positively proportional to the first consistency information , and positively proportional to the second consistency information . After determining the score of each piece of the candidate reply information, a piece of candidate reply information having the highest score therein is determined as the target reply information.", + "p0059": "In the present embodiment, the piece of candidate reply information, which has a greater preceding text semantic relevance, a higher degree of first consistency, and a higher degree of second consistency, is determined as the target reply information, and thus the accuracy of determining the target reply information may be improved.", + "p0060": "In some application scenarios, as shown in FIG. 4 , the method for determining information includes:", + "p0061": "In the first step, inputting historical dialogue/whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information into a BERT model, to obtain word vectors of the whole preceding text information and word vectors of each piece of candidate reply information in the plurality pieces of candidate reply information output from the BERT model.", + "p0062": "In particular, the above step includes: acquiring , where U = represents the whole preceding text information,", + "p0064": "represents an i-th sentence in the preceding text information,", + "p0066": "represents an N-th word in the i-th sentence in the whole preceding text information,", + "p0068": "represents the i-th candidate reply information, and", + "p0070": "represents the N-th word in the i-th candidate reply information; the whole preceding text information U and each piece of candidate reply information r i are concatenated and then input into a pre-trained BERT model, and the word vectors of the whole preceding text information and the word vectors of the candidate reply information output from the BERT model are obtained:", + "p0072": "where, BERT returns output from the last layer of the BERT model. < ; > represents concatenation/splicing of the two sequences preceding and succeeding the semicolon, H U represents the word vectors of the whole preceding text information U, and H r i represents the word vector of the candidate reply information r i . In addition, a total vector", + "p0074": "of semantic information of the whole preceding text information may be obtained based on the BERT model.", + "p0075": "In the second step, after obtaining the word vectors of the whole preceding text information and the word vectors of the piece of candidate reply information, performing fine-grained reply comparison based on the two.", + "p0076": "In particular, the step includes: first, for each piece of candidate reply information, calculating a word-level attention between the word vectors H U of the whole preceding text information and the word vectors H r i of the piece of candidate reply information, and obtaining a similarity matrix between the two:", + "p0078": "where,", + "p0080": "where, \u2299 represents elementwise multiplication between the two matrices.", + "p0082": "is the vector representation of an m-th word in r i , and", + "p0084": "is a model parameter.", + "p0086": "represents the similarity between the m-th word in r i and the n-th word in r j .", + "p0087": "Secondly, based on the above similarity matrix A r i,j , obtaining the difference information between r i and r j , and comparison information between r i and r j is defined as:", + "p0090": "where, H r i,j represents a similar part between r i and r j , and H r i,j represents a different part between r i and r j , i.e., the difference information between r i and r j .", + "p0091": "Finally, after acquiring the difference information between the current r i and each r j in the plurality pieces of candidate reply information, combining the above difference information to obtain a differentiated representation H\u0303 r i between r i and all pieces of the rest candidate reply information, i.e., the total difference information:", + "p0095": "where, E r i is an intermediate variable of the computational equation; represents a splicing operation, i.e., splicing sequences in ; \u03c3 represents the Dirac \u03b4 function; and W 2 , W 3 and b 2 represent model parameters.", + "p0096": "In the third step, performing logical consistency reasoning on the whole preceding text: based on the differentiated representation H\u0303 r i between r i and all of the rest candidate reply information, determining the consistency information H\u0302 h_i between the candidate reply information r i and the whole preceding text information .", + "p0097": "In particular, the above step including:", + "p0106": "where, W 4 , W 5 , W 6 , W 7 , W 8 , W 9 , b 4 are model parameters; A h_i and A i_h are word-level attention matrices between the whole preceding text information U and the candidate reply information r i , which focus on different perspectives, i.e., attention models used to obtain the two types of data focus on different information; the attention model used to obtain A h_i focuses on the whole preceding text information related to the reply information, and the attention model used to obtain A i_h focuses on the reply information related to the whole preceding text information. MaxPooling denotes a maximum pooling operation, SoftMax denotes a normalized exponential function, and Relu denotes a linear rectification function; H h_i is a reply-aware preceding text representation, and H i_h is a preceding-text-aware reply representation; and g hi denotes a gate mechanism value obtained by fusing H h_i and H i_h based on the gate mechanism.", + "p0107": "In the fourth step: enhancing consistency of overall information of the interlocutor: determining the consistency information H\u0302 s_i between the candidate reply information r i and the target preceding text information , based on the differentiated representation H\u0303 r i between r i and all of the other candidate reply information, where, the target preceding text information refers to the whole preceding text information given by the interlocutor to which the piece of candidate reply information belongs .", + "p0112": "where, A s _i and A i_s are word-level attention matrices between the target preceding text information and the candidate reply information r i , which focus on different perspectives, i.e., attention models used to obtain the two types of data focus on different information; the attention model used to obtain A s _i focuses on the target preceding text information, and the attention model used to obtain A i_s focuses on the reply information related to the target preceding text information; mark T denotes transpose of the matrix; H s _i denotes a reply-aware speaker representation, is the vector representation of the whole preceding text information, and this vector representation is related to the reply information; H i _s represents a speaker-aware reply representation, is the vector representation of the reply, and this vector representation is related to the target preceding text information; and W 10 , W 11 , W 12 , W 13 are model parameters.", + "p0113": "Then, the consistency information H\u0302 s_i between the candidate reply information r i and the target preceding text information may be determined:", + "p0118": "where, W 14 , W 15 , b 5 , b 6 are model parameters; and g si denotes a gate mechanism value obtained by fusing H s _i and H i _s based on the gate mechanism.", + "p0119": "In the fifth step, first, data of the three, i.e., the total vector", + "p0121": "of the semantics of the whole preceding text information, the consistency information H\u0302 h_i between the candidate reply information r i and the whole preceding text information, and the consistency information H\u0302 s_i between the candidate reply information r i and the target preceding text information, are spliced to obtain inference information H i of the candidate reply information r i :", + "p0123": "Secondly, based on the inference information, the score P of the candidate reply information r i is predicted:", + "p0125": "where, W 16 and b 6 are model parameters, set M is an identifier of the candidate reply information in the plurality pieces of candidate reply information other than the candidate reply information r i ; and R denotes a set of the candidate reply information.", + "p0126": "In this application scenario, the above model parameters may be obtained based on model training, and a system for training the model may be shown in FIG. 5 , and steps of training the model include: acquiring sample dialogue data, where the sample dialogue data includes dialogue history and candidate replies, filtering the sample dialogue data to exclude incomplete data and redundant data to obtain training data; inputting the training data into a knowledge transfer predictor to obtain word vectors of each piece of training data, inputting the training data processed by the knowledge transfer predictor into a knowledge-aware generator to obtain knowledge-aware information , and determining the target reply information based on the knowledge-aware information. A loss function of the model may be defined as:", + "p0128": "where, \u03bb is a hyperparameter, \u03b8 is all trainable parameters, N is a size of the training data in the dataset, and", + "p0130": "is the actual reply information. The model is optimized based on the loss function, and when a loss value of the loss function meets a preset threshold, it may be determined that the training of the model is complete, and the model parameters may be obtained based on the trained model.", + "p0131": "In this application scenario, the structure of the system for determining target reply information may include:", + "p0132": "Preceding text encoding module: given the whole preceding text information and the current candidate reply information, the BERT model is used to encode the input information to obtain word-level word vector representations of the given whole preceding text information and the current candidate reply information, and to obtain a semantic representation of the whole preceding text information and a semantic representation of the candidate reply information.", + "p0133": "Fine-grained comparison module: given the word vectors of the whole preceding text information and the candidate reply information, the total difference information between the current candidate reply information and other candidate reply information may be obtained by calculating the similarity matrix between the current candidate reply information and the other candidate replay information. Then, a biphasic matching mechanism is applied to compare the word vectors of the whole preceding text information with the total difference information in both positive and negative directions respectively, to obtain the reply-aware preceding text representation and the preceding-text-aware reply representation, and then the representations are combined via the gate mechanism, to obtain the first consistency information between the whole preceding text information and the current candidate reply information.", + "p0134": "Another biphasic matching mechanism is applied to compare the word vectors of the speaker's preceding text with the total difference information to obtain the reply-aware speaker's preceding text representation and the speaker-aware reply representation, and then the two representations are combined via the gate mechanism, to obtain the second consistency information between the speaker's preceding text and the current candidate reply information.", + "p0135": "Prediction module: given the semantic representation of the whole preceding text information and the semantic representation of the current candidate reply information, the first consistency information between the whole preceding text information and the candidate reply information, and the second consistency information between the speaker's whole preceding text information and the current candidate reply information, the three may be spliced and then input into a softmax network to obtain a final score of the candidate reply. A training objective of the model is to maximize a probability value of correct replies.", + "p0136": "An example of the method for determining information in this application scenario is: Dialogue history information is \"A: It's already 6:30, shall we start preparing dinner now? B: But I don't want to cook, I'm too tired to cook every day. A: Why don't we go out for eating? There is a new Chinese restaurant on Third Avenue. Xiaoming went there yesterday and said it was very good. B: Really? What types of food do they have? You know, I can't eat spicy food. A: Don't worry, the chef is Cantonese. I know that Cantonese food is one of your favorite dishes. B: That's great, do you know how to get there? A: I don't know where it is, I just know it's on Third Avenue. Don't worry, I think we can find it. B: But I don't want to walk, it's too hot outside. A: Well, why not have Xiaoming pick us up and we can treat him for dinner. B: Good idea\".", + "p0137": "The candidate reply information includes the following options: \"1) Since Sichuan food is your favorite dish, why don't we go over there after work? 2) Okay, I know where the restaurant is. 3) Okay, let's go! 4) Okay, we can go eat your favorite Sichuan food after school.\"", + "p0138": "Taking option 1) as an example, the whole preceding text information is spliced with option 1), and the BERT model is used to obtain the vector representation of each word, and obtain a same-dimension vector representation of the whole semantic information.", + "p0139": "Comparing between Option 1) and Option 2) and the difference is ; the difference between Option 1) and Option 3) is ; and the difference between Option 1) and Option 4) is . The difference contents are then integrated to obtain the total difference information between option 1) and all the other options. The total difference information is matched with the whole preceding text information, and it was found that Sichuan food and Cantonese food do not match, and there is no word regarding after work in the text, so option 1) would have a relatively low score in terms of whole preceding text consistency. Finally, the difference content is matched with Speaker A's historical preceding text, and it was found that Speaker A had said something about the other person favorites Cantonese food, it would therefore enhance the comparison of this type of information with the difference content, which found that Sichuan food and Cantonese food do not match. Therefore, the speaker's preceding text consistency score of option 1) is also relatively low .", + "p0140": "As above, the difference content between option 2) and option 1), 3), and 4) is compared and a match score is calculated, since \"I know where the restaurant is\" contradicts the original text in which A said \"I don't know where it is but I believe we can find it\", the match score is low . Option 3) is compared with option 1), 2), 4), since there is no contradictory information, the score is relatively high. Option 4) is compared with option 1), 2), 3) and the match score is calculated, since \"favorite Sichuan food\" contradicts the original text in which A said \"your favorite Cantonese food\", the match score is low. Finally, the scores of the four options may be ranked, and then option 3) is determined as the target reply information.", + "p0141": "With further reference to FIG. 6 , as an implementation of the method shown in the above figures, an embodiment of the present disclosure provides an apparatus for determining information, and the apparatus embodiment corresponds to the method embodiments shown in FIG. 2 and FIG. 3 . The apparatus may be applied to various electronic devices.", + "p0142": "As shown in FIG. 6 , the apparatus for determining information in the present embodiment, including: an acquisition unit 601, a first determining unit 602, a second determining unit 603, and a third determining unit 604. The acquisition unit is configured to acquire whole preceding text information and a plurality pieces of candidate reply information for replying to the whole preceding text information. The first determining unit is configured to determine, for each piece of candidate reply information in the plurality pieces of candidate reply information, total difference information between the piece of candidate reply information and other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information. The second determining unit is configured to determine consistency information between the total difference information and the whole preceding text information as first consistency information between the piece of candidate reply information and the whole preceding text information. The third determining unit is configured to determine target reply information from the plurality pieces of candidate reply information, based on the first consistency information corresponding to each piece of candidate reply information.", + "p0143": "In some embodiments, the first determining unit includes: a first acquisition module, configured to acquire word vectors of each piece of candidate reply information; a first determining module, configured to determine, for each piece of candidate reply information, based on a similarity between the word vectors of the piece of candidate reply information and the word vectors of each piece of the other candidate reply information, difference information between the piece of candidate reply information and each piece of the other candidate reply information; and a second determining module, configured to determine the difference information between the piece candidate reply information and all of the other candidate reply information as the total difference information between the piece of candidate reply information and the other candidate reply information.", + "p0144": "In some embodiments, the second determining unit includes: a second acquisition module, configured to acquire word vectors of the whole preceding text information; and a third determining module, configured to determine consistency information between the total difference information and the word vectors of the whole preceding text information as the first consistency information between the piece of candidate reply information and the whole preceding text information.", + "p0145": "In some embodiments, the apparatus further includes: a fourth determining unit, configured to determine consistency information between the total difference information and target preceding text information as second consistency information between the piece of candidate reply information and the target preceding text information, where the target preceding text information is information in the whole preceding text information that belongs to a same interlocutor as the piece of candidate reply information; and the third determining unit includes: a fourth determining module, configured to determine the target reply information from the plurality pieces of candidate reply information, based on the first consistency information and the second consistency information corresponding to each piece of candidate reply information.", + "p0146": "In some embodiments, the total difference information is determined based on the similarity between the word vectors of the piece of candidate reply information and the word vectors of the other candidate reply information in the plurality pieces of candidate reply information other than the piece of candidate reply information, and the fourth determining unit includes: a third acquisition module, configured to acquire word vectors of the target preceding text information; and a fifth determining module, configured to determine consistency information between the total difference information and the word vectors of the target preceding text information as the second consistency information between the piece of candidate reply information and the target preceding text information.", + "p0147": "In some embodiments, the fourth determining module includes: a scoring module, configured to determine, for each piece of candidate reply information, a score of the piece of candidate reply information, on basis of the first consistency information corresponding to the piece of candidate reply information, the second consistency information corresponding to the piece of candidate reply information, and third consistency information between semantics of the piece of candidate reply information and semantics of the whole preceding text information; and a selection module, configured to determine the target reply information from the plurality pieces of candidate reply information, based on the score of each piece of candidate reply information.", + "p0148": "The units in the apparatus 600 described above correspond to the steps in the method described with reference to FIG. 2 and FIG. 3 . The operations, features and technical effects achieved by the method for determining information described above are thus equally applicable to the apparatus 600 and the units contained therein, and detailed description thereof will be omitted.", + "p0149": "According to an embodiment of the present disclosure, an electronic device and a readable storage medium are provided.", + "p0150": "As shown in FIG. 7 , is a block diagram of an electronic device of the method for determining information according to embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other appropriate computers. The electronic device may alternatively represent various forms of mobile apparatuses such as personal digital assistant, a cellular telephone, a smart phone, a wearable device and other similar computing apparatuses. The parts shown herein, their connections and relationships, and their functions are only as examples, and not intended to limit implementations of the present disclosure as described and/or claimed herein.", + "p0151": "As shown in FIG. 7 , the electronic device includes: one or more processors 701, a memory 702, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses, and may be installed on a common motherboard or in other methods as needed. The processor may process instructions executed within the electronic device, including instructions stored in or on the memory to display graphic information of GUI on an external input/output apparatus . In other embodiments, a plurality of processors and/or a plurality of buses may be used together with a plurality of memories if desired. Similarly, a plurality of electronic devices may be connected, and the devices provide some necessary operations . In FIG. 7 , one processor 701 is used as an example.", + "p0152": "The memory 702 is a non-transitory computer readable storage medium provided by an embodiment of the present disclosure. The memory stores instructions executable by at least one processor, so that the at least one processor performs the method for determining information provided by embodiments of the present disclosure. The non-transitory computer readable storage medium of the present disclosure stores computer instructions for causing a computer to perform the method for determining information provided by embodiments of the present disclosure.", + "p0153": "The memory 702, as a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions/modules corresponding to the method for determining information in embodiments of the present disclosure. The processor 701 executes the non-transitory software programs, instructions, and modules stored in the memory 702 to execute various functional applications and data processing of the server, that is, to implement the method for determining information in the foregoing method embodiments.", + "p0154": "The memory 702 may include a storage program area and a storage data area, where the storage program area may store an operating system and at least one function required application program; and the storage data area may store data created by the use of the electronic device according to the method for processing parking, etc. In addition, the memory 702 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 702 may optionally include memories remotely provided with respect to the processor 701, and these remote memories may be connected to the electronic device for extracting video clip. Examples of the above network include but are not limited to the Internet, intranet, local area network, mobile communication network, and combinations thereof.", + "p0155": "The electronic device of the method for determining information may further include: an input apparatus 703 and an output apparatus 704. The processor 701, the memory 702, the input apparatus 703, and the output apparatus 704 may be connected through a bus or in other methods. In Fig. 7 , connection through a bus is used as an example.", + "p0156": "The input apparatus 703 may receive input digital or character information, and generate key signal inputs related to user settings and function control of the electronic device of the method for processing parking, such as touch screen, keypad, mouse, trackpad, touchpad, pointing stick, one or more mouse buttons, trackball, joystick and other input apparatuses. The output apparatus 704 may include a display device, an auxiliary lighting apparatus , a tactile feedback apparatus , and the like. The display device may include, but is not limited to, a liquid crystal display , a light emitting diode display, and a plasma display. In some embodiments, the display device may be a touch screen.", + "p0157": "Various embodiments of the systems and technologies described herein may be implemented in digital electronic circuit systems, integrated circuit systems, dedicated ASICs , computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and/or interpreted on a programmable system that includes at least one programmable processor. The programmable processor may be a dedicated or general-purpose programmable processor, and may receive data and instructions from a storage system, at least one input apparatus, and at least one output apparatus, and transmit the data and instructions to the storage system, the at least one input apparatus, and the at least one output apparatus.", + "p0158": "These computing programs include machine instructions of the programmable processor and may use high-level processes and/or object-oriented programming languages, and/or assembly/machine languages to implement these computing programs. As used herein, the terms \"machine readable medium\" and \"computer readable medium\" refer to any computer program product, device, and/or apparatus ) used to provide machine instructions and/or data to the programmable processor, including machine readable medium that receives machine instructions as machine readable signals. The term \"machine readable signal\" refers to any signal used to provide machine instructions and/or data to the programmable processor.", + "p0159": "In order to provide interaction with a user, the systems and technologies described herein may be implemented on a computer, the computer has: a display apparatus for displaying information to the user or LCD monitor); and a keyboard and a pointing apparatus , and the user may use the keyboard and the pointing apparatus to provide input to the computer. Other types of apparatuses may also be used to provide interaction with the user; for example, feedback provided to the user may be any form of sensory feedback ; and any form may be used to receive input from the user.", + "p0160": "The systems and technologies described herein may be implemented in a computing system that includes backend components , or a computing system that includes middleware components , or a computing system that includes frontend components , or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication . Examples of the communication network include: local area networks , wide area networks , the Internet, and blockchain networks.", + "p0161": "The computer system may include a client and a server. The client and the server are generally far from each other and usually interact through the communication network. The relationship between the client and the server is generated by computer programs that run on the corresponding computer and have a client-server relationship with each other.", + "p0162": "It should be understood that the various forms of processes shown above may be used to reorder, add, or delete steps. For example, the steps described in embodiments of the present disclosure may be performed in parallel, sequentially, or in different orders. As long as the desired results of the technical solution disclosed in embodiments of the present disclosure can be achieved, no limitation is made herein.", + "p0163": "The above specific embodiments do not constitute limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions may be made according to design requirements and other factors. Any modification, equivalent replacement and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.", + "features": { + "F1": "A method for determining information by acquiring whole preceding text information and a plurality of pieces of candidate reply information for replying to the whole preceding text information.", + "F2": "Determining total difference information for each piece of candidate reply information in the plurality of pieces of candidate reply information by comparing it against other candidate reply information.", + "F3": "Determining first consistency information by assessing consistency between the total difference information and the whole preceding text information for each piece of candidate reply information.", + "F4": "Selecting target reply information from the plurality of candidate reply information based on the first consistency information corresponding to each piece.", + "F5": "Acquiring word vectors for each piece of candidate reply information to calculate similarity and derive total difference information.", + "F6": "Determining difference information based on similarity between word vectors of candidate reply information and other candidate reply information.", + "F7": "Acquiring word vectors of the whole preceding text information to determine first consistency information.", + "F8": "Determining second consistency information between total difference information and target preceding text information, where target text belongs to the same interlocutor as the reply.", + "F9": "Determining target reply information based on first and second consistency information.", + "F10": "Computing a score for each piece of candidate reply information using first, second, and third consistency information, with third reflecting semantic consistency.", + "F11": "An apparatus for determining information, comprising units to acquire text information, determine difference and consistency information, and select target replies.", + "F12": "An acquisition unit in the apparatus configured to acquire whole preceding text and candidate reply information.", + "F13": "A first determining unit configured to calculate total difference information among candidate replies.", + "F14": "A second determining unit to compute consistency information between total difference and whole preceding text.", + "F15": "A third determining unit to identify target reply information based on consistency metrics.", + "F16": "A first acquisition module in the apparatus for obtaining word vectors of candidate replies.", + "F17": "Modules within the first determining unit that compute similarity-based difference and total difference information.", + "F18": "A second acquisition module to acquire word vectors of whole preceding text information.", + "F19": "A third determining module to assess consistency between total difference and whole text vectors.", + "F20": "A fourth determining unit to derive second consistency information with specific target text.", + "F21": "A third acquisition module to obtain word vectors for target preceding text information.", + "F22": "A scoring module to evaluate candidate replies based on multiple consistency values.", + "F23": "A selection module to determine target replies based on computed scores.", + "F24": "An electronic device with a processor and memory for executing the method described in claims 1-6.", + "F25": "A non-transitory computer-readable medium storing instructions to execute the method from claims 1-6.", + "F26": "A computer program product implementing the method from claims 1-6 when executed by a processor." + } + } + }, + { + "FAN": "77017157", + "CPC": [ + "G06F-016/00", + "G06F-016/245", + "G06F-016/287", + "G06F-016/903", + "G06F-017/30/424", + "G06F-017/30/601", + "H04B-007/185/06" + ], + "Content": { + "title": "Systems and methods for flexible access of internal data of an avionics system", + "pa01": "The present disclosure provides systems and methods that enable flexible access of internal data of an avionics system . Aspects of the present disclosure enable enhanced, flexible and robust recording of flight test data, testing, debugging, and analyzing in-service problems. One example system includes a Configuration Tool that generates a Loadable Configuration File and a Configuration File based on user selections, structure of data in Data Stores, platform information, processor information, and Receiving Component requirements; a Data Access Component that subscribes to data in the Data Stores based on the Loadable Configuration File and outputs the data; and a Receiving Component receives the data, and decodes the data based at least in part on the Configuration File. Aspects of the present disclosure provide the ability to change which data gets output by the avionics system without the need to recompile and recertify the avionics system software.", + "c-en-0001": "A method to provide flexible access to internal data of an avionics system, the method comprising: receiving, by a Data Access Component of the avionics system, a Loadable Configuration File that specifies one or more data Topics selected by a user, the Data Access Component implemented by one or more processors; accessing, by the Data Access Component, one or more Data Stores that contain the internal data of the avionics system to retrieve data associated with the one or more data Topics specified by the Loadable Configuration File; outputting, by the Data Access Component, the retrieved data to a Receiving Component; providing, by a Configuration Tool, a user interface that enables the user to select the one or more data Topics from a set of available Topics, the Configuration Tool implemented by one or more processors; receiving, by the Configuration Tool via the user interface, a user input that selects the one or more data Topics from the set of available Topics; generating, by the Configuration Tool, the Loadable Configuration File based at least in part on the received user input, wherein the Loadable Configuration File specifies the one or more data Topics and one or more of an offset and a size associated with each Topic; and providing the Loadable Configuration File to the Data Access Component.", + "c-en-0002": "The method of claim 1 , further comprising: generating, by the Configuration Tool, a Configuration File based at least in part on the received user input; and providing the Configuration File to the Receiving Component.", + "c-en-0003": "The method of claim 2 , further comprising: decoding, by the Receiving Component, the data output by the Data Access Component based on the Configuration File, the Receiving Component implemented by one or more processors.", + "c-en-0004": "The method of claim 1 , further comprising: obtaining, by the Configuration Tool, a Parsed Available Data File that lists the set of available Topics; and generating, by the Configuration Tool based on the Parsed Available Data File, the user interface that enables the user to select the one or more data Topics from the set of available Topics.", + "c-en-0005": "The method of claim 4 , further comprising: parsing an Application Source Code associated with the avionics system to obtain the Parsed Available Data File.", + "c-en-0006": "The method of claim 1 , wherein accessing, by the Data Access Component, the one or more Data Stores to retrieve the data associated with the one or more data Topics comprises subscribing, by the Data Access Component, to the Data Stores that contain the data associated with the one or more data Topics.", + "c-en-0007": "The method of claim 1 , wherein accessing, by the Data Access Component, the one or more Data Stores to retrieve the data associated with the one or more data Topics comprises: receiving, by the Data Access Component, a request for a particular set the data associated with the one or more data Topics from the Receiving Component; and in response to receipt of the request, accessing, by the Data Access Component, the one or more Data Stores to retrieve the data associated with the one or more data Topics.", + "c-en-0008": "The method of claim 1 , wherein: the Loadable Configuration File defines a plurality of different Topic groups; and accessing, by the Data Access Component, the one or more Data Stores to retrieve the data associated with the one or more data Topics comprises: determining, by the Data Access Component based on the Loadable Configuration File, a rate at which each of the plurality of Topic groups should be output; and accessing, by the Data Access Component, the one or more Data Stores according to the respective determined rates to retrieve each Topic group at its respective rate.", + "c-en-0009": "The method of claim 1 , further comprising: receiving, by a User Interface Painter of the avionics system, the Loadable Configuration File that specifies one or more data Topics selected by the user, the User Interface Painter implemented by one or more processors; accessing, by the User Interface Painter, the one or more Data Stores that contain the internal data of the avionics system to retrieve the data associated with the one or more data Topics specified by the Loadable Configuration File; and outputting, by the User Interface Painter, the retrieved data to a Display Unit.", + "c-en-0010": "The method of claim 9 , further comprising: receiving, by a User Interface Handler of the avionics system, one or more events from a Display Unit, wherein the one or more events correspond to user input received at the Display Unit; and providing, by the User Interface Handler, an interface for the events to the Data Access Component.", + "c-en-0011": "A method to provide flexible access to internal data of an avionics system, the method comprising: generating, via a Configuration Tool, a Loadable Configuration File and a Configuration File based on user selections, structure of data in Data Stores, platform information, processor information, and Receiving Component requirements; subscribing, via a Data Access Component, to data in the Data Stores based on the Loadable Configuration File; outputting, via the Data Access Component, the data received from the Data Stores; receiving, via a Receiving Component, the data output by the Data Access Component; decoding, via the Receiving Component, the data based at least in part on the Configuration File; providing, via the Configuration Tool, a user interface that enables a user to select one or more data Topics from a set of available Topics; and receiving, via the Configuration Tool, a user input that selects the one or more data Topics from the set of available Topics; wherein generating, via the Configuration Tool, the Loadable Configuration File and the Configuration File based on user selections comprises generating, via the Configuration Tool, the Loadable Configuration File and the Configuration File based on the one or more data Topics selected by the user input.", + "c-en-0012": "The method of claim 11 , further comprising: obtaining, via the Configuration Tool, a Parsed Available Data File that lists the set of available Topics; and generating, via the Configuration Tool based on the Parsed Available Data File, the user interface that enables the user to select the one or more data Topics from the set of available Topics.", + "c-en-0013": "The method of claim 12 , further comprising: parsing an Application Source Code associated with the avionics system to obtain the Parsed Available Data File.", + "c-en-0014": "A system, comprising: a Configuration Tool implemented by one or more processors, wherein the Configuration Tool generates a Loadable Configuration File and a Configuration File based on user selections and at least one of a structure of data in one or more Data Stores, platform information, processor information, and one or more Receiving Component requirements; a Data Access Component implemented by one or more processors, wherein the Data Access Component subscribes to data in the Data Stores based on the Loadable Configuration File and outputs the data; and a Receiving Component implemented by one or more processors, wherein the Receiving Component receives the data output by the Data Access Component and decodes the data output by the Data Access Component based at least in part on the Configuration File; wherein the Configuration Tool, obtains a Parsed Available Data File that describes a set of available Topics; provides a user interface that enables a user to select one or more of the Topics; and generates the Loadable Configuration File based at least in part on the Topics selected by the user.", + "c-en-0015": "The system of claim 14 , further comprising: a User Interface Painter implemented by one or more processors, wherein the User Interface Painter subscribes to data in the Data Stores based on the Loadable Configuration File and outputs the data to a Display Unit.", + "c-en-0016": "The system of claim 15 , further comprising the Display Unit.", + "c-en-0017": "The system of claim 14 , further comprising: a User Interface Handler that provides an interface for events received from a Display Unit.", + "p0001": "PRIORITY CLAIM", + "p0002": "The present application claims priority to and the benefit of U.S. Provisional Patent Application Ser. No. 62/296,615, filed Feb. 18, 2016, titled \u201cSystems and Methods for Flexible Access of Internal Data of an Avionics System.\u201d", + "p0003": "Incorporation by Reference", + "p0004": "U.S. Provisional Patent Application Ser. No. 62/296,615, filed Feb. 18, 2016, titled \u201cSystems and Methods for Flexible Access of Internal Data of an Avionics System\u201d is hereby incorporated by reference in its entirety. Commonly owned and copending U.S. patent application Ser. No. 14/486,325, filed Sep. 15, 2015, titled \u201cMechanism And Method For Communicating Between A Client And A Server By Accessing Message Data In A Shared Memory;\u201d Ser. No. 14/486,336, filed Sep. 15, 2015, titled \u201cMechanism And Method For Accessing Data In A Shared Memory;\u201d and Ser. No. 14/585,519, filed Dec. 30, 2015, titled \u201cMethod And System For Ensuring Integrity Of Critical Data\u201d are hereby incorporated by reference in their entireties.", + "p0005": "FIELD OF THE INVENTION", + "p0006": "The present subject matter relates generally to avionics systems and, more particularly, systems and methods for flexible access of internal data of an avionics system.", + "p0007": "BACKGROUND OF THE INVENTION", + "p0008": "Modern aircraft employ complex Flight Management Systems that handle and control large amounts of important data for the aircraft. Presently, in order to access internal data from the Flight Management System, an interface that is defined at compile time in the software is required. However, due to certification requirements and safety concerns, changing or altering this access interface necessitates additional testing and recertification. As a result of the additional testing and recertification requirements, the access interface can be difficult and costly to change. Often these interfaces will have undergone many changes throughout the course of a development program, but are too costly to change once the system is in service.", + "p0009": "BRIEF DESCRIPTION OF THE INVENTION", + "p0010": "Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.", + "p0011": "One example aspect of the present disclosure is directed to a method to provide flexible access to internal data of an avionics system. The method includes receiving, by a Data Access Component of the avionics system, a Loadable Configuration File that specifies one or more data Topics selected by a user. The Data Access Component is implemented by one or more processors. The method includes accessing, by the Data Access Component, one or more Data Stores that contain the internal data of the avionics system to retrieve data associated with the one or more data Topics specified by the Loadable Configuration File. The method includes outputting, by the Data Access Component, the retrieved data to a Receiving Component.", + "p0012": "Another example aspect of the present disclosure is directed to a method to provide flexible access to internal data of an avionics system. The method includes generating, via a Configuration Tool, a Loadable Configuration File and a Configuration File based on user selections, structure of data in Data Stores, platform information, processor information, and Receiving Component requirements. The method includes subscribing, via a Data Access Component, to data in the Data Stores based on the Loadable Configuration File. The method includes outputting, via the Data Access Component, the data received from the Data Stores. The method includes receiving, via a Receiving Component, the data output by the Data Access Component. The method includes decoding, via the Receiving Component, the data based at least in part on the Configuration File.", + "p0013": "Another example aspect of the present disclosure is directed to a system. The system includes a Configuration Tool implemented by one or more processors. The Configuration Tool generates a Loadable Configuration File and a Configuration File based on user selections and at least one of a structure of data in one or more Data Stores, platform information, processor information, and one or more Receiving Component requirements. The system includes a Data Access Component implemented by one or more processors. The Data Access Component subscribes to data in the Data Stores based on the Loadable Configuration File and outputs the data. The system includes a Receiving Component implemented by one or more processors. The Receiving Component receives the data output by the Data Access Component and decodes the data output by the Data Access Component based at least in part on the Configuration File.", + "p0014": "Variations and modifications can be made to these example aspects of the present disclosure.", + "p0015": "These and other features, aspects and advantages of various embodiments will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the related principles.", + "p0016": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0017": "Detailed discussion of embodiments directed to one of ordinary skill in the art are set forth in the specification, which makes reference to the appended figures, in which:", + "p0018": "FIG. 1 depicts a block diagram of an example system for flexible access of internal data of an avionics system according to example embodiments of the present disclosure;", + "p0019": "FIG. 2 depicts a block diagram of an example system for flexible access of internal data of an avionics system according to example embodiments of the present disclosure;", + "p0020": "FIG. 3 depicts a block diagram of an example data access framework according to example embodiments of the present disclosure;", + "p0021": "FIG. 4 depicts a top down schematic view of an example aircraft according to example embodiments of the present disclosure; and", + "p0022": "FIG. 5 depicts a block diagram of an example computing device according to example embodiments of the present disclosure.", + "p0023": "DETAILED DESCRIPTION OF THE INVENTION", + "p0024": "Reference now will be made in detail to embodiments of the invention, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the invention, not limitation of the invention. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the scope or spirit of the invention. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present invention covers such modifications and variations as come within the scope of the appended claims and their equivalents.", + "p0025": "Example aspects of the present disclosure are directed to systems and methods that enable flexible access to internal data of an avionics system . In particular, as noted in the Background section, an interface that is defined at compile time in the software is typically required to access internal data from a Flight Management System. However, due to certification requirements and safety concerns, changing or altering the access interface necessitates additional testing and recertification. As a result of such additional testing and recertification requirements, the access interface can be difficult and costly to change. Accordingly, one advantage of the disclosed subject matter is to provide a method and system that enables flexible access of internal data of an avionics system .", + "p0026": "Aspects of the present disclosure provide a technique to select, customize, or otherwise flexibly define data that is output by the avionics system during runtime. In particular, in some implementations, systems of the present disclosure can include a Configuration Tool that enables a system operator to make selections of which data is thereafter output by or otherwise retrieved from the avionics system during runtime.", + "p0027": "As one example, in some implementations, the Application Source Code of the avionics system can be parsed to create a Parsed Available Data File that lists data Topics available for subscription or access. The Parsed Available Data File can further include information about one or more parameters associated with each data Topic, size of each data Topic, one or more offsets associated with each data Topic, endianness of the Data Stores, byte alignment, or other information regarding the available data Topics and their anticipated storage structure.", + "p0028": "The Configuration Tool can use the Parsed Available Data File to provide an interactive graphical user interface that allows the user to make selections of which data Topics are output by or otherwise retrieved from the avionics system during runtime. The user interface can enable the user to make various other selections regarding preferences for a number of other data settings or parameters, including, for example, output frequency for each data Topic, selected parameters for each data Topic, groups and sub-groups of Topics or Topic parameters, etc.", + "p0029": "Based on such user selections, the Configuration Tool can generate a Loadable Configuration File that instructs or otherwise informs a Data Access Component of the avionics system regarding the identity of the data Topics selected by the user. Further, in some implementations, in addition identifying the Topics selected by the user, the Loadable Configuration can include additional information for each identified Topic, such as, for example, offsets, size, parameters, etc. In some implementations, the Loadable Configuration File can specify certain data groups that can be toggled during runtime of the avionics system.", + "p0030": "In some implementations, in addition to the user selections, the Configuration Tool can generate the Loadable Configuration file based at least in part on other information such as, without limitation, a structure of data in one or more Data Stores of the avionics system, platform information, processor information, and/or one or more requirements associated with a Receiving Component that receives the selected data output by the avionics system. In addition, the Configuration Tool can also generate a Configuration File that can be used to decode the data output by the Data Access Component, as will be discussed further below.", + "p0031": "The Data Access Component can subscribe to data in the Data Stores based on the Loadable Configuration File. In particular, the Data Access Component can output data from the subscribed Data Stores during runtime of the avionics system. In some implementations, the Data Access Component can output the data as a stream output to a Receiving Component. The Receiving Component can receive the data and decode the data based at least in part on the Configuration File received from the Configuration Tool.", + "p0032": "The Receiving Component can be any device, program, application, or other system component for which the system operator desires to provide selected data from the avionics system. As examples, the Receiving Component can be a simulation system, a data recorder , an onboard maintenance system, an in-flight air-to-ground data transmission system, or any other device, system, or component for which receipt of particular user-selected avionics data is desired, including components internal to the avionics system.", + "p0033": "Thus, aspects of the present disclosure provide a technique to customize or otherwise define data that is output by the avionics system during runtime. Furthermore, aspects of the present disclosure enable enhanced, flexible, and robust recording of flight test data, testing, debugging, and analyzing of in-service problems.", + "p0034": "Although example aspects of the present disclosure are discussed with reference to avionics systems, the subject matter described herein can be used with or applied to provide flexible access to internal data of other systems, vehicles, machines, industrial or mechanical assets, or components without deviating from the scope of the present disclosure.", + "p0035": "With reference now to the FIGS., example aspects of the present disclosure will be discussed in further detail.", + "p0036": "FIG. 1 depicts a block diagram of an example system 100 for flexible access of internal data of an avionics system 102 according to example embodiments of the present disclosure. The avionics system 102 can include, but is not limited to, a Flight Management System.", + "p0037": "In some implementations, the avionics system 102 executes, operates, or otherwise runs on a Framework that provides data and time management . For example, the Framework can manage data , and threads or timing called processing functions. The processing functions can communicate with each other via the Topics. In some implementations, the Framework enables access to Topics in a thread-safe way. In some implementations, the Framework provides data management on a publish/subscription basis.", + "p0038": "One example visualization of the Framework is provided in FIG. 3 . Additional details regarding the Framework are included in commonly owned and copending U.S. patent application Ser. No. 14/486,325, filed Sep. 15, 2015, titled \u201cMechanism And Method For Communicating Between A Client And A Server By Accessing Message Data In A Shared Memory;\u201d Ser. No. 14/486,336, filed Sep. 15, 2015, titled \u201cMechanism And Method For Accessing Data In A Shared Memory;\u201d and Ser. No. 14/585,519. Use of the example Framework by the avionics system 102 is provided as one example only and the present disclosure is not limited by or to this example context. Other techniques, systems, or platforms that enable data access and/or management by the avionics system 102 can be used in addition or alternatively to the Framework.", + "p0039": "Referring again to FIG. 1 , the avionics system 102 includes a Data Access Component 104 . The Data Access Component 104 executes, operates or otherwise runs based on a Loadable Configuration File 106 . For example, the Loadable Configuration File 106 can include instructions for the Data Access Component 104 to output predetermined data every 2 seconds.", + "p0040": "Also illustrated in FIG. 1 is a Configuration Tool 108 . The Configuration Tool 108 enables a user to select data to be output from the Data Access Component 104 , and the frequency of said output. For example, the Configuration Tool 108 can include a program or graphic user interface through which the user picks data from a predetermined set of data, and a frequency of output for the selected data.", + "p0041": "In some implementations, the Configuration Tool 108 has prior knowledge of how data is maintained in the Data Stores 110 , the platform environment, the processor, etc. Based on user input via the user interface, the Configuration Tool 108 can generate the Loadable Configuration File 106 that instructs the Data Access Component 104 to access data or Topics, and output the data based on selections made by the user.", + "p0042": "In some implementations, the Data Access Component 104 parses the Loadable Configuration File 106 at startup and subscribes to a set of data or list of Topics based on a list or instructions contained in the Loadable Configuration File 106 . In some implementations, the Data Access Component 104 can subscribe to the data or Topics through the use of string literals and generic data types, so that the data specification is not needed at run time, thereby making the interface more flexible.", + "p0043": "In some implementations, in addition to specifying Topic names, the Loadable Configuration File 106 can also specify particular data entries within a Topic. The Loadable Configuration File 106 can provide instructions regarding how many bytes are to be read. For example, an offset can describe a number of bytes from the beginning of the Topic where a particular data item starts; a size can describe a number of bytes that a particular data item occupies; and a type can define how to interpret those bytes.", + "p0044": "The requested data specified by the Loadable Configuration File 106 can be grouped so a sub-group of data can be requested. Each group can output data at a defined rate and the Data Access Component 104 can manage run rates. This allows for limiting the amount of data output simultaneously by splitting it across multiple groups. The consumer of the data can then send requests to the Data Access Component to toggle the output as needed.", + "p0045": "As one example, at each cycle the Data Access Component 104 can read data of the required Topics through a Framework-provided pointer and can extract individual data items based on the offsets and sizes defined in the Loadable Configuration File 106 . The data can be output in human readable format in which case the types can be used for casting. Thus, the Data Access Component 104 can perform conversion for console output.", + "p0046": "Additionally or alternatively, the output data can be serialized into a byte stream and output as a Stream Output 114 . For instance, the Stream Output 114 can include either ARINC 429 or ARINC 664, for example, where it is preferable to limit the data output to the maximum payload . Additionally or alternatively, pagination can be employed, for example, where multiple transmissions get indices as well as the maximum number of transmissions to expect for the receiver. The option of grouping can also be provided where not all data is needed contiguously.", + "p0047": "Furthermore, the Data Access Component 104 can also provide protection mechanisms to avoid \u201cdraining\u201d the system by requesting too much data. For example, Cyclic Redundancy Checks, public and private keys, or other encryption or security technology can be used, as described further below. These protection mechanisms can be in addition to mechanisms in the Configuration Tool 108 . The Data Access Component 104 can also check the input file and make sure it meets a set of predetermined criteria . For example, the Data Access Component 104 can ensure that the number of bytes requested per output group does not exceed a threshold value.", + "p0048": "Further illustrated in FIG. 1 is a Receiving Component 116 . The Receiving Component 116 can include a data recorder , an onboard maintenance system, or any other component or system for which customized data output is desired, including components both internal to or external to the avionics system 102 .", + "p0049": "In one example, the Receiving Component 116 can be a data communication/data downlink application. Therefore, a change in the message set does not require a rebuild of the application. For example, the Receiving Component 116 can be a datalink application/partition of a Flight Management System.", + "p0050": "The Receiving Component 116 can receive the Stream Output 114 from the Data Access Component 104 . Additionally or alternatively, the Receiving Component 116 can request or trigger actions from the Data Access Component 104 . For example, the Receiving Component 116 can trigger a data output from the Data Access Component 104 via a Request Queue 118 that is serviced by the Data Access Component 104 . In some implementations, the Request Queue 118 can be used to send particular requests to enable and/or disable data output based on the different groups or sub-groups defined in the Loadable Configuration File 106 .", + "p0051": "The Receiving Component 116 can trigger data output from, or contract, the Data Access Component 104 in a plurality of ways, including but not limited to periodically , on demand , or based on an event. For example, the Receiving Component 116 can trigger an output of data when an aircraft reaches cruising altitude. Events used as triggers can be predefined, and the data to be output can be configurable.", + "p0052": "The Receiving Component 116 can enable robust and flexible access to aircraft data during testing. For example, partition-level testing may require access to data/Topics. Previously, for example, a hard-coded Bus would be required to access the desired information/data. However, in view of the disclosures herein, the Receiving Component 116 can trigger a data output from the Data Access Component 104 and decode the received data.", + "p0053": "In particular, the Configuration Tool 108 can further provide a Configuration File 120 to the Receiving Component 116 that enables the Receiving Component 116 to decode data obtained, acquired, or otherwise received from the Data Access Component 104 . For example, data received from the Data Access Component 104 can be a set of serialized data . The Receiving Component 116 can decode the serialized data based on information contained in the Configuration File 120 .", + "p0054": "In some implementations, the Configuration Tool 108 generates the Configuration File 120 based at least in part on prior knowledge of the platform environment, the Data Stores 110 , the processor, avionics system software version, etc., which enables the Configuration tool 108 to instruct the Receiving Component 116 how to decode received data from the Data Access Component 104 .", + "p0055": "The Receiving Component 116 can decode the received data before or after recording. For example, the Receiving Component 116 can decode the received data while in flight or during post-processing. In addition, the Receiving Component 116 can ensure that it has an up-to-date Configuration File 120 to decode the data stream based on version information included in or associated with a Configuration File 120 .", + "p0056": "The Configuration Tool 108 provides an easy-to-use interface that enables a user to set, choose, or otherwise select criteria for a Config File 120 . The Configuration Tool 108 generates the Configuration File 120 based on awareness of the structure of the data in the Data Stores 110 and requirements for the Receiving Component 116 to decode received information. The Configuration Tool 108 generates both the Loadable Configuration File 106 for the Data Access Component 104 and the Configuration File 120 for the Receiving Component 116 .", + "p0057": "In some implementations, the Configuration Tool 108 or other system component parses a set of Application Source Code 122 to provide a list of all available Topics. The Application Source Code 122 can be the source code of the avionics system 102 . In particular, the Application Source Code 122 can describe or otherwise contain the structure of the Data Stores 110 or other information about the structure and/or operation of the avionics system 102 and/or the Framework. Parsing the Application Source Code 122 can result in a Parsed Available Data File 124 which contains a list of all available Topics and/or other information .", + "p0058": "In some implementations, in addition to the Application Source Code 122 , the Configuration Tool 108 can analyze or otherwise use one or more binary object files of the avionics system 102 to determine the memory layout of the system 102 . For example, a compiler can optionally create a context clause . This context clause can contain tables describing the memory layout. The Configuration Tool 108 can use the context clause as an input in addition to the Source Code 122 .", + "p0059": "Based on the target processor architecture , the Configuration Tool 108 is also aware of byte alignment and endianness to adjust offsets in the Loadable Config File 120 . The endianness information is used by the Receiving Component 116 to decode the data stream. The Configuration Tool 108 may also account for units of each data item either through meta data in the Application Source Code 122 or through the type definition name . Additionally or alternatively, the Configuration Tool 108 can enable the user to update, change, or otherwise add the meta data. For example, where the Configuration Tool 108 cannot infer the meta data, the user can add meta data that will be maintained for future use.", + "p0060": "Thus, the Parsed Available Data File 124 can be a representation of the Application Source Code 122 that the Configuration Tool 108 can use to generate both the Loadable Config File 106 and the Config File 120 for the Receiving Component 116 . This intermediate data file 124 can be used to provide the Configuration Tool 108 to a third party and allow them to create their own Loadable Config Files 106 while limiting scope and visibility to the Application Source Code 122 .", + "p0061": "Creation of rogue Config Files, for example, where someone does a trial and error test to extract undesired data, or attempts to crash the software, can be prevented by applying a Cyclic Redundancy Check to the Loadable Config Files 106 and/or Config Files 120 or using any other form of a hash or electronic signature.", + "p0062": "In some implementations, the Loadable Config File 106 and the Config File 120 can have maximum size limits. Likewise, the contents of the file can have a group number limit. This can influence the memory and request interface sizes internal to the Data Access Component 104 .", + "p0063": "Each of the Configuration Tool 108 , the Data Access Component 104 , and the Receiving Component 116 can be implemented by one or more computing devices. As used herein, a \u201ccomputing device\u201d can include one or more processors and a memory. The one or more processors can be can be any suitable processing device and can be one processor or a plurality of processors that are operatively connected. The memory can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory can store instructions which are executed by the processor to perform operations.", + "p0064": "In one particular non-limiting example, the Configuration Tool 108 is implemented by a first computing device that is located in a ground station; the avionics system 102 is implemented by a second computing device that is on-board an aircraft; and the Receiving Component is implemented by a third computing device that is located on-board the aircraft. However, the previous distribution is provided as one example only. Many other and different distributions of computing functionality can be used.", + "p0065": "In some implementations, each of the Configuration Tool 108 , the Data Access Component 104 , and the Receiving Component 116 correspond to or otherwise include computer logic utilized to provide desired functionality. Thus, each of the Configuration Tool 108 , the Data Access Component 104 , and the Receiving Component 116 can be implemented in hardware, application specific circuits, firmware and/or software controlling a general purpose processor. In one implementation, each of the Configuration Tool 108 , the Data Access Component 104 , and the Receiving Component 116 correspond to program code files stored on a storage device, loaded into memory and executed by a processor or can be provided from computer program products, for example computer executable instructions, that are stored in a tangible computer-readable storage medium such as RAM, hard disk or optical or magnetic media. In some implementations, each of the Data Stores 110 can include some form of non-transitory computer-readable storage media, including, for example, volatile and/or non-volatile memory, databases, etc.", + "p0066": "Referring now to FIG. 2 , illustrated is a second example system 200 for flexible access of internal data of an avionics system 202 according to example embodiments of the present disclosure. The avionics system 202 can include, but is not limited to, a Flight Management System.", + "p0067": "The system 200 of FIG. 2 is largely similar to the system 100 of FIG. 1 , as indicated by the inclusion of many identically numbered components. For example, the system 200 includes the Configuration Tool 108 , the Data Access Component 104 , the Receiving Component 116 , and many other identically numbered components which are discussed with reference to FIG. 1 .", + "p0068": "However, in addition to a Data Access Component 104 , the avionics system 202 of FIG. 2 further includes a User Interface Painter 204 and a User Interface Handler 206 . In some implementations, the User Interface Painter 204 and/or the User Interface Handler 206 can be table-based or otherwise table driven.", + "p0069": "The User Interface Painter 204 can also receive the Loadable Configuration File 106 from the Configuration Tool 108 . Based on the Topics identified by the Loadable Configuration File 106 , the User Interface Painter 204 can subscribe to certain Data Stores 110 upon power up or initial execution of the system 202 . The User Interface Painter 204 can also provide file protections in a manner similar to that provided by the Data Access Component 104 . In addition, the User Interface Painter 204 can perform conversions for units, resolutions, or other parameters.", + "p0070": "The User Interface Painter 204 can output or otherwise provide the data obtained from the Data Stores 110 to a Display Unit 208 , for example, as illustrated at 211 . In some implementations, the Display Unit 208 can be a multifunction control display unit. The Display Unit 208 can include one or more screens for displaying data and can also include one or more buttons or other controls for receiving user input. Thus, the User Interface Painter 204 and the Display Unit 208 can cooperatively operate to enable the display of user-selected data, as guided by the Loadable Configuration File 106 .", + "p0071": "In some implementations, the Display Unit 208 can operate according to the ARINC 661 standard. Thus, the data 211 provided from the User Interface Painter 204 to the Display Unit 208 can be formatted into ARINC 661 blocks.", + "p0072": "Similar to the Configuration File 120 , the Display Unit 208 can receive a UADF Configuration File 210 that was generated by the Configuration Tool 108 . The UADF Configuration File 210 can enable the Display Unit 208 to decode the data 211 received from the User Interface Painter 204 . For example, the UADF Configuration File 210 can be a ARINC 661 UADF Configuration File.", + "p0073": "The User Interface Handler 206 can also subscribe to one or more Data Stores 110 upon power up. In particular, in some implementations, the User Interface Painter 204 and the User Interface Handler 206 can be portions of the same component or otherwise cooperatively operate.", + "p0074": "The User Interface Handler 206 can provide an interface for incoming events from the Display Unit 208 . For example, user input received by the Display Unit 208 can be provided to the User Interface Handler 206 , as illustrated at 212 . In some implementations, the data 212 provided from the Display Unit 208 to the User Interface Handler 206 can be formatted into ARINC 661 blocks.", + "p0075": "Thus, the Loadable Configuration File 106 can be used to customize or other select particular data to be provided from the avionics system 202 to both a Receiving Component 116 and a Display Unit 208 . Other components that are not specifically illustrated can receive customized data outputs as well. In some implementations, the Configuration Tool 108 enables separate customization and user selection of different data to be respectively output to the Receiving Component 116 and the Display Unit 208 . In particular, two or more Loadable Configuration Files 106 can be generated which provide or reflect different user selections of customized output data respectively for the Receiving Component 116 and the Display Unit 208 , and potentially other components as well.", + "p0076": "Each of the User Interface Painter 204 and the User Interface Handler 206 can be implemented by one or more computing devices. In some implementations, each of the User Interface Painter 204 and the User Interface Handler 206 correspond to or otherwise include computer logic utilized to provide desired functionality. Thus, each of the User Interface Painter 204 and the User Interface Handler 206 can be implemented in hardware, application specific circuits, firmware and/or software controlling a general purpose processor. In one implementation, each of the User Interface Painter 204 and the User Interface Handler 206 correspond to program code files stored on a storage device, loaded into memory and executed by a processor or can be provided from computer program products, for example computer executable instructions, that are stored in a tangible computer-readable storage medium such as RAM, hard disk or optical or magnetic media.", + "p0077": "Turning now to FIG. 3 , illustrated is an example Framework 300 for communication of data in an avionics system in accordance with various aspects described herein. As discussed, the Framework 300 provides data and time management. For example, the Framework 300 can manage data and can manage threads or timing called processing functions. For instance, the Framework 300 can manage a first processing function that runs every 200 ms and performs a first task, and at the same time a second processing function that runs every 1 s and performs a second task. The Framework 300 can include a set of Data Stores or Topics and a set of mail slots. The Framework 300 enables accessing data from the Data Stores and mail slots.", + "p0078": "Referring to FIG. 3 , a set of processing functions are in communication with, linked to, or otherwise connected to the Framework 300 . The processing functions can execute, operate or otherwise run at flexible or rigid times. The processing functions 302 can access data from a set of Data Stores included in a Data Store Repository 308 maintained by the Framework 300 . For example, the processing function 302 is illustrated as accessing or otherwise interacting with the Data Stores 304 and 306 of the Data Store Repository 308 .", + "p0079": "The Framework 300 enables access of data items by name . By knowing the name that will be used in the Configuration File, the Configuration Tool can specify the name of the data in which the user is interested. The Data Access Component can parse the Configuration File to subscribe to the appropriate Data Stores.", + "p0080": "In particular, whenever the system is restarted, the Data Access Component can load the Configuration File and subscribe/re-subscribe to the desired data. Additionally or alternatively, data groups can be specified to toggle the data being output at runtime and the Data Access Component 104 can parse and subscribe to all potential groups.", + "p0081": "Referring now to FIG. 4 , an example aircraft 8 is shown having a fuselage 10 and at least one turbine engine, shown as a left engine system 12 and a right engine system 14 . The left and right engine systems 12 , 14 may be substantially identical. While turbine engines 12 , 14 are illustrated, the aircraft 8 may include fewer or additional engine systems, or alternative propulsion engine systems, such as propeller-based engines.", + "p0082": "The aircraft 8 is shown further comprising a plurality of sensors, systems, and components, collectively referred to as line-replaceable units 18 , and at least one server 20 or computing unit, shown as two flight management systems, or flight control computers, located proximate to each other, near the nose of the aircraft 8 . At least one of the servers 20 may further include memory 22 .", + "p0083": "The LRUs 18 and servers 20 may be communicatively interconnected by transmission and/or communication lines defining a data communications network 24 , traversing at least a portion of the aircraft 8 . Additional LRUs 18 may be included. While a server 20 is described, embodiments of the invention may include any computing system, flight computer, or display system displaying data from multiple systems.", + "p0084": "The memory 22 may include random access memory , flash memory, or one or more different types of portable electronic memory, etc., or any suitable combination of these types of memory. The LRUs 18 and/or servers 20 may be operably coupled with the memory 22 such that the LRUs 18 and/or servers 20 , or any computer programs or processes thereon, may access at least a portion of the memory 22 .", + "p0085": "The aircraft 8 shown in FIG. 4 is merely a schematic representation of one example embodiment of the invention, and used to illustrate that a plurality of LRUs 18 and servers 20 may be located throughout the aircraft 8 . The exact location of the LRUs 18 and servers 20 are not germane to the embodiments of the invention. Additionally, more or fewer LRUs 18 and/or servers 20 may be included in embodiments of the invention.", + "p0086": "The communications network 24 is illustrated as a bus, but may include a number of data communication connectors and interfaces, for example, Ethernet or fiber-optic cables, and routing and/or switching components, to facilitate the communicative interconnection between the LRUs and Servers 20 . Furthermore, the configuration and operation of the communications network 24 may be defined by a common set of standards or regulations applicable to particular aircraft environments. For example, the communications network 24 on an aircraft 8 may be defined by, and/or configured according to, the ARINC 664 standard, or ARINC 653 standard.", + "p0087": "Referring now to FIG. 5 , a block diagram of an example computing device 500 is illustrated. In some implementations, the computing device 500 can be referred to as a controller or a microcontroller. The computing device 500 can be stand-alone or can be embedded into a computing system, such as the avionics system 102 of FIG. 1 .", + "p0088": "The computing device 500 can include one or more processors 502 and a memory 504 . The one or more processors 502 can be can be any suitable processing device and can be one processor or a plurality of processors that are operatively connected. The memory 504 can include one or more non-transitory computer-readable storage mediums, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 504 can store data 506 , such as data Topics in Data Stores. The memory 504 can also store instructions 508 which are executed by the processor 502 to perform operations described by the present disclosure.", + "p0089": "It is readily apparent that an advantage of the present disclosure is that it enables truly configurable and dynamic access to the data without the need to recompile and recertify the aircraft system . The Configuration Tool includes a proprietary data file that allows any licensed user to generate a Loadable Config File in a simple and robust fashion.", + "p0090": "The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. One of ordinary skill in the art will recognize that the inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, server processes discussed herein may be implemented using a single server or multiple servers working in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.", + "p0091": "Although specific features of various embodiments may be shown in some drawings and not in others, this is for convenience only. In accordance with the principles of the present disclosure, any feature of a drawing may be referenced and/or claimed in combination with any feature of any other drawing.", + "p0092": "This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims. Accordingly, the invention is not to be limited to any single embodiment or implementation, but rather can be construed in breadth, spirit and scope in accordance with the appended claims.", + "features": { + "F1": "A method to provide flexible access to internal data of an avionics system comprises receiving a Loadable Configuration File that specifies one or more data Topics selected by a user.", + "F2": "The Data Access Component accesses one or more Data Stores to retrieve data associated with the one or more data Topics specified by the Loadable Configuration File.", + "F3": "The Data Access Component outputs the retrieved data to a Receiving Component.", + "F4": "A Configuration Tool provides a user interface enabling the selection of one or more data Topics from a set of available Topics.", + "F5": "The Configuration Tool generates the Loadable Configuration File based on the received user input, specifying one or more data Topics and optionally an offset and a size for each Topic.", + "F6": "The method further comprises generating a Configuration File based on the user input and providing it to the Receiving Component.", + "F7": "The Receiving Component decodes the data output by the Data Access Component based on the Configuration File.", + "F8": "The Configuration Tool obtains a Parsed Available Data File listing the set of available Topics and generates the user interface based on this file.", + "F9": "Parsing the Application Source Code associated with the avionics system obtains the Parsed Available Data File.", + "F10": "Accessing the Data Stores to retrieve data comprises the Data Access Component subscribing to the Data Stores.", + "F11": "Accessing the Data Stores to retrieve the data involves receiving a request for the data from the Receiving Component and accessing the Data Stores in response.", + "F12": "The Loadable Configuration File defines multiple Topic groups, and the Data Access Component determines and accesses the Data Stores at respective rates for each Topic group.", + "F13": "A User Interface Painter receives the Loadable Configuration File, accesses the Data Stores, and outputs the retrieved data to a Display Unit.", + "F14": "A User Interface Handler receives events from a Display Unit and provides an interface for the events to the Data Access Component.", + "F15": "A system comprises a Configuration Tool, Data Access Component, and Receiving Component, each implemented by processors, with defined roles in generating, subscribing, and decoding data based on configuration files.", + "F16": "The Configuration Tool generates a Loadable Configuration File and Configuration File based on user selections, data structure, platform, and processor information.", + "F17": "The system includes a User Interface Painter that subscribes to data in the Data Stores and outputs it to a Display Unit.", + "F18": "The system comprises a User Interface Handler providing an interface for events from a Display Unit." + } + } + }, + { + "FAN": "106232152", + "CPC": [ + "G06F-003/0483", + "G06F-003/0484/2", + "G06F-003/0484/5", + "G06F-016/00", + "G06F-016/955", + "G06F-016/9562", + "G06F-016/957", + "G06F-2203/04806" + ], + "Content": { + "title": "Method for implementing collection function of browser, and readable medium and electronic device", + "pa01": "This application relates to a method for implementing a favorites function of a browser, a readable medium, and an electronic device. The method includes: detecting a favorites selection operation of a user, where a favorites object selected by the favorites selection operation is at least partial displayed content on a tab currently displayed on the browser; storing related data of the at least partial displayed content in response to the favorites selection operation, where the related data is used to display the at least partial displayed content; and displaying a favorites tab in response to a view operation of the user, where the at least partial displayed content is displayed on the favorites tab. In the technical solutions of this application, partial resources in which the user is interested on the tab of the browser can be added to favorites, to meet a requirement of the user for adding the partial resources on the tab of the browser to favorites. The favorites tab intuitively displays a plurality of resources that are added to favorites, to help the user quickly find resources in which the user is interested on the favorites tab and improve use experience when the user uses the favorites function of the browser.", + "c-en-0001": "A method for implementing a favorites function of a browser, applied to an electronic device on which the browser is installed, wherein the method comprises: detecting a favorites selection operation of a user, wherein a favorites object selected by the favorites selection operation is at least partial displayed content on a tab currently displayed on the browser; storing related data of the at least partial displayed content in response to the favorites selection operation, wherein the related data is used to display the at least partial displayed content; and displaying a favorites tab in response to a view operation of the user, wherein the at least partial displayed content is displayed on the favorites tab.", + "c-en-0002": "The method according to claim 1, wherein the storing related data of the at least partial displayed content in response to the favorites selection operation comprises: in response to the favorites selection operation, determining the related data of the at least partial displayed content that is selected by the user to add to favorites; storing the related data of the at least partial displayed content as a resource package corresponding to the at least partial displayed content; determining a storage path of the resource package corresponding to the at least partial displayed content and a name of the resource package; and storing the storage path of the resource package corresponding to the at least partial displayed content and the name of the resource package.", + "c-en-0003": "The method according to claim 1 or 2, further comprising: in response to the favorites selection operation, determining a basic parameter corresponding to the at least partial displayed content that is selected by the user to add to favorites, wherein the basic parameter is for describing a favorites type corresponding to the at least partial displayed content and at least one of location information and size information of the at least partial displayed content when the at least partial displayed content is displayed on the favorites tab; and storing the basic parameter corresponding to the at least partial displayed content.", + "c-en-0004": "The method according to claim 3, wherein the basic parameter comprises a favorites type parameter corresponding to the at least partial displayed content; and the displaying a favorites tab in response to a view operation of the user comprises: in response to the view operation of the user, obtaining the stored storage path of the resource package corresponding to the at least partial displayed content, the stored name of the resource package, and the stored basic parameter corresponding to the partial displayed content; and when it is determined, based on the obtained basic parameter corresponding to the partial displayed content, that the partial displayed content needs to be added to favorites and displayed, displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content.", + "c-en-0005": "The method according to claim 4, wherein the basic parameter further comprises a location parameter and a size parameter that correspond to the at least partial displayed content; and the displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content comprises: determining, based on the basic parameter corresponding to the partial displayed content, a first location and a first size of the at least partial displayed content on the favorites tab when the at least partial displayed content is displayed on the favorites tab; finding the related data of the at least partial displayed content based on the storage path of the resource package and the name of the resource package; and displaying the at least partial displayed content at the first location of the favorites tab based on the found related data of the at least partial displayed content, wherein a size of the partial displayed content displayed on the favorites tab is the first size.", + "c-en-0006": "The method according to claim 5, further comprising: in response to a movement operation of the user, moving the at least partial displayed content displayed on the favorites tab from the first location to a second location.", + "c-en-0007": "The method according to claim 5 or 6, further comprising: in response to a scaling operation of the user, adjusting the at least partial displayed content displayed on the favorites tab from the first size to a second size.", + "c-en-0008": "The method according to any one of claims 1 to 7, further comprising: in response to a hiding operation of the user, hiding the at least partial displayed content displayed on the favorites tab.", + "c-en-0009": "The method according to any one of claims 1 to 8, wherein the related data of the at least partial displayed content comprises: the at least partial displayed content or a uniform resource locator corresponding to the at least partial displayed content.", + "c-en-0010": "The method according to claim 9, wherein the related data of the at least partial displayed content further comprises: a display style file of the at least partial displayed content and/or a script file of the at least partial displayed content, wherein the display style file of the at least partial displayed content is for describing a display style of the at least partial displayed content on the currently displayed tab; and the script file of the at least partial displayed content is for describing operation logic of the at least partial displayed content on the currently displayed tab.", + "c-en-0011": "The method according to any one of claims 1 to 10, wherein the at least partial displayed content comprises one or more of a video, an audio, a picture, and a text.", + "c-en-0012": "An apparatus for implementing a favorites function of a browser, comprising: a detecting module, configured to detect a favorites selection operation of a user, wherein a favorites object selected by the favorites selection operation is at least partial displayed content on a tab currently displayed on the browser; a first response module, configured to store related data of the at least partial displayed content in response to the favorites selection operation, wherein the related data is used to display the at least partial displayed content; and a second response module, configured to display a favorites tab in response to a view operation of the user, wherein the at least partial displayed content is displayed on the favorites tab.", + "c-en-0013": "The apparatus according to claim 12, wherein the first response module stores the related data of the at least partial displayed content in response to the favorites selection operation in the following manner: in response to the favorites selection operation, determining the related data of the at least partial displayed content that is selected by the user to add to favorites; storing the related data of the at least partial displayed content as a resource package corresponding to the at least partial displayed content; determining a storage path of the resource package corresponding to the at least partial displayed content and a name of the resource package; and storing the storage path of the resource package corresponding to the at least partial displayed content and the name of the resource package.", + "c-en-0014": "The apparatus according to claim 12 or 13, further comprising: a third response module, configured to: in response to the favorites selection operation, determine a basic parameter corresponding to the at least partial displayed content that is selected by the user to add to favorites, wherein the basic parameter is for describing a favorites type corresponding to the at least partial displayed content and at least one of location information and size information of the at least partial displayed content when the at least partial displayed content is displayed on the favorites tab; and store the basic parameter corresponding to the at least partial displayed content.", + "c-en-0015": "The apparatus according to claim 14, wherein the basic parameter comprises a favorites type parameter corresponding to the at least partial displayed content; and the second response module displays the favorites tab in response to the view operation of the user in the following manner: in response to the view operation of the user, obtaining the stored storage path of the resource package corresponding to the at least partial displayed content, the stored name of the resource package, and the stored basic parameter corresponding to the partial displayed content; and when it is determined, based on the obtained basic parameter corresponding to the partial displayed content, that the partial displayed content needs to be added to favorites and displayed, displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content.", + "c-en-0016": "The apparatus according to claim 15, wherein the basic parameter further comprises a location parameter and a size parameter that correspond to the at least partial displayed content; and the second response module displays the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content in the following manner: determining, based on the basic parameter corresponding to the partial displayed content, a first location and a first size of the at least partial displayed content on the favorites tab when the at least partial displayed content is displayed on the favorites tab; finding the related data of the at least partial displayed content based on the storage path of the resource package and the name of the resource package; and displaying the at least partial displayed content at the first location of the favorites tab based on the found related data of the at least partial displayed content, wherein a size of the partial displayed content displayed on the favorites tab is the first size.", + "c-en-0017": "The apparatus according to claim 16, further comprising: a fourth response module, configured to: in response to a movement operation of the user, move the at least partial displayed content displayed on the favorites tab from the first location to a second location.", + "c-en-0018": "The apparatus according to claim 16 or 17, further comprising: a fifth response module, configured to: in response to a scaling operation of the user, adjust the at least partial displayed content displayed on the favorites tab from the first size to a second size.", + "c-en-0019": "The apparatus according to any one of claims 12 to 18, further comprising: a sixth response module, configured to: in response to a hiding operation of the user, hide the at least partial displayed content displayed on the favorites tab.", + "c-en-0020": "The apparatus according to any one of claims 12 to 19, wherein the related data of the at least partial displayed content comprises: the at least partial displayed content or a uniform resource locator corresponding to the at least partial displayed content.", + "c-en-0021": "The apparatus according to claim 20, wherein the related data of the at least partial displayed content further comprises: a display style file of the at least partial displayed content and/or a script file of the at least partial displayed content, wherein the display style file of the at least partial displayed content is for describing a display style of the at least partial displayed content on the currently displayed tab; and the script file of the at least partial displayed content is for describing operation logic of the at least partial displayed content on the currently displayed tab.", + "c-en-0022": "The apparatus according to any one of claims 12 to 21, wherein the at least partial displayed content comprises one or more of a video, an audio, a picture, and a text.", + "c-en-0023": "A computer-readable storage medium, wherein the computer-readable storage medium stores instructions, and when the instructions are executed on an electronic device, the electronic device is enabled to perform the method for implementing a favorites function according to any one of claims 1 to 11.", + "c-en-0024": "A computer program product, wherein the computer program product comprises instructions, and when the instructions are executed by one or more processors, the instructions are used to implement the method for implementing a favorites function according to any one of claims 1 to 11.", + "c-en-0025": "An electronic device, comprising: a memory, configured to store instructions; and one or more processors, wherein when the instructions are executed by the one or more processors, the processor performs the method for implementing a favorites function according to any one of claims 1 to 11.", + "p0001": "This application claims priority to Chinese Patent Application No. 202210152026.2, filed with the China National Intellectual Property Administration on February 18, 2022 and entitled \"METHOD FOR IMPLEMENTING FAVORITES FUNCTION OF BROWSER, READABLE MEDIUM, AND ELECTRONIC DEVICE\", which is incorporated herein by reference in its entirety.", + "p0002": "TECHNICAL FIELD", + "p0003": "This application relates to the field of computer technologies, and in particular, to a method for implementing a favorites function of a browser, a readable medium, and an electronic device.", + "p0004": "BACKGROUND", + "p0005": "As Internet content becomes increasingly rich, a browser may display a plurality of resources such as a text, an image, a video, and an audio download from a web page server. There are more cases in which a user uses the browser to access the Internet to browse content in which the user is interested. For example, the user searches for a meaning of a technical term, watches a video, listens to a song, searches for lyrics, and reads a novel through the browser.", + "p0006": "Some of these web pages are web pages in which the user is interested and hopes to access again in the future. Usually, browsers such as IE \u2122 and Chrome \u2122 allow the user to add these web pages to favorites folders of the browser. In this way, after the user opens the browser next time, the user does not need to manually input uniform resource locators of these web pages that have been added to favorites or find a web page in which the user is interested by using a search engine, but can directly access these web pages by using the favorites folders. Introduction of the favorites folders greatly facilitates an operation of accessing a specific website by the user, but there are still some problems.", + "p0007": "For example, some existing browsers usually add all content of a web page selected by the user to favorites, but in some cases, the user is interested in only partial content of the web page. However, the existing browsers cannot implement a function of adding the partial content of the web page to favorites, and cannot meet a requirement of the user for adding partial resources in the web page to favorites. For example, content corresponding to a web page whose website address is url1 is displayed in a display interface 110 of a browser of a computer 100 shown in FIG. 1A . After the user taps a favorites control 111, the browser adds url1 corresponding to all content of the web page to a local favorites folder 112 shown in FIG. 1B , and a name is \"favorites content 1\", but cannot add partial content of the web page to favorites.", + "p0008": "For another example, in some existing browsers, names of web pages that are added to favorites in a favorites folder are sequentially arranged in a list form from top to bottom. The favorites folder occupies less space in a display interface of the browser, and a layout is inappropriate. When there is a large amount of content that is added to favorites, it is difficult for the user to quickly find content in which the user is interested from the favorites folder. As a result, search efficiency is low. For example, in the local favorites folder 112 of the browser shown in FIG. 1B , the local favorites folder 112 is located in a narrow area 1 on a left side in the display interface of the browser. Names of web pages added by the browser to favorites, for example, favorites content 1 to X in the figure, can only be sequentially displayed in the list form from top to bottom. When the user needs to search for specific content from a large amount of content added to favorites in the local favorites folder 112, the user needs to drag a control 113 to search the large amount of content added to favorites in the local favorites folder 112 one by one with naked eyes. This layout manner of the favorites folder is not conducive to the user to quickly find target content, and user experience is poor.", + "p0009": "SUMMARY", + "p0010": "In view of this, embodiments of this application provide a method for implementing a favorites function of a browser, a readable medium, and an electronic device. In the technical solutions of this application, when a favorites selection operation of a user is detected, related data of a favorites object selected by the favorites selection operation is stored, and a favorites tab is displayed in response to a view operation of the user, where the at least partial displayed content is displayed on the favorites tab. Therefore, an electronic device is enabled to add, based on a requirement of the user, partial resources in which the user is interested on the tab of the browser to favorites, to meet the requirement of the user for adding the partial resources on the tab of the browser to favorites. In addition, the favorites tab intuitively displays a plurality of resources that are added to favorites, to help the user quickly find resources in which the user is interested in the plurality of resources displayed on the favorites tab, so as to improve use experience when the user uses the favorites function of the browser.", + "p0011": "According to a first aspect, this application provides a method for implementing a favorites function of a browser, applied to an electronic device on which the browser is installed. The method includes:", + "p0012": "detecting a favorites selection operation of a user, where a favorites object selected by the favorites selection operation is at least partial displayed content on a tab currently displayed on the browser; storing related data of the at least partial displayed content in response to the favorites selection operation, where the related data is used to display the at least partial displayed content; and displaying a favorites tab in response to a view operation of the user, where the at least partial displayed content is displayed on the favorites tab.", + "p0013": "When the favorites selection operation of the user is detected, the related data of the favorites object selected by the favorites selection operation is stored, and the favorites tab is displayed in response to the view operation of the user, where the at least partial displayed content is displayed on the favorites tab. Therefore, an electronic device is enabled to add, based on a requirement of the user, partial resources in which the user is interested on the tab of the browser to favorites, to meet the requirement of the user for adding the partial resources on the tab of the browser to favorites. In addition, the favorites tab intuitively displays a plurality of resources that are added to favorites, to help the user quickly find resources in which the user is interested in the plurality of resources displayed on the favorites tab, so as to improve use experience when the user uses the favorites function of the browser.", + "p0014": "According to the first aspect, in a possible implementation of the first aspect, the storing related data of the at least partial displayed content in response to the favorites selection operation includes:", + "p0015": "in response to the favorites selection operation, determining the related data of the at least partial displayed content that is selected by the user to add to favorites; storing the related data of the at least partial displayed content as a resource package corresponding to the at least partial displayed content; determining a storage path of the resource package corresponding to the at least partial displayed content and a name of the resource package; and storing the storage path of the resource package corresponding to the at least partial displayed content and the name of the resource package.", + "p0016": "Therefore, when it is determined that the partial displayed content needs to be displayed, the related data of the at least partial displayed content is obtained based on the stored storage path of the resource package and the stored name of the resource package, so that the at least partial displayed content is displayed on the favorites tab based on the obtained related data of the at least partial displayed content.", + "p0017": "According to the first aspect, in a possible implementation of the first aspect, the method further includes:", + "p0018": "in response to the favorites selection operation, determining a basic parameter corresponding to the at least partial displayed content that is selected by the user to add to favorites, where the basic parameter is for describing a favorites type corresponding to the at least partial displayed content and at least one of location information and size information of the at least partial displayed content when the at least partial displayed content is displayed on the favorites tab; and storing the basic parameter corresponding to the at least partial displayed content.", + "p0019": "Therefore, when it is determined that the partial displayed content needs to be displayed, the favorites type corresponding to the partial displayed content is determined based on the stored basic parameter. When it is determined, based on the favorites type, that the partial displayed content needs to be added to favorites and displayed, the location information and the size information of the at least partial displayed content when being displayed on the favorites tab are determined based on the stored basic parameter, to display the partial displayed content of a corresponding size at a corresponding location on the favorites tab.", + "p0020": "According to the first aspect, in a possible implementation of the first aspect, the basic parameter includes a favorites type parameter corresponding to the at least partial displayed content; and", + "p0021": "the displaying a favorites tab in response to a view operation of the user includes: in response to the view operation of the user, obtaining the stored storage path of the resource package corresponding to the at least partial displayed content, the stored name of the resource package, and the stored basic parameter corresponding to the partial displayed content; and when it is determined, based on the obtained basic parameter corresponding to the partial displayed content, that the partial displayed content needs to be added to favorites and displayed, displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content.", + "p0022": "According to the first aspect, in a possible implementation of the first aspect, the basic parameter further includes a location parameter and a size parameter that correspond to the at least partial displayed content; and", + "p0023": "the displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content includes: determining, based on the basic parameter corresponding to the partial displayed content, a first location and a first size of the at least partial displayed content on the favorites tab when the at least partial displayed content is displayed on the favorites tab; finding the related data of the at least partial displayed content based on the storage path of the resource package and the name of the resource package; and displaying the at least partial displayed content at the first location of the favorites tab based on the found related data of the at least partial displayed content, where a size of the partial displayed content displayed on the favorites tab is the first size.", + "p0024": "According to the first aspect, in a possible implementation of the first aspect, the method further includes: in response to a movement operation of the user, moving the at least partial displayed content displayed on the favorites tab from the first location to a second location. In this way, the user can adjust locations of favorites content on the favorites tab based on a requirement of the user, to improve user experience.", + "p0025": "According to the first aspect, in a possible implementation of the first aspect, the method further includes: in response to a scaling operation of the user, adjusting the at least partial displayed content displayed on the favorites tab from the first size to a second size, so that the user can adjust sizes of favorites content on the favorites tab based on the requirement of the user, to improve user experience.", + "p0026": "According to the first aspect, in a possible implementation of the first aspect, the method further includes: in response to a hiding operation of the user, hiding the at least partial displayed content displayed on the favorites tab, so that the user can hide some favorites content on the favorites tab based on the requirement of the user, or redisplay some hidden content on the favorites tab for processing, to meet different requirements of the user.", + "p0027": "According to the first aspect, in a possible implementation of the first aspect, the related data of the at least partial displayed content includes: the at least partial displayed content or a uniform resource locator corresponding to the at least partial displayed content.", + "p0028": "According to the first aspect, in a possible implementation of the first aspect, the related data of the at least partial displayed content further includes:", + "p0029": "a display style file of the at least partial displayed content and/or a script file of the at least partial displayed content, where the display style file of the at least partial displayed content is for describing a display style of the at least partial displayed content on the currently displayed tab; and the script file of the at least partial displayed content is for describing operation logic of the at least partial displayed content on the currently displayed tab.", + "p0030": "According to the first aspect, in a possible implementation of the first aspect, the at least partial displayed content includes one or more of a video, an audio, a picture, or a text.", + "p0031": "According to a second aspect, this application provides an apparatus for implementing a favorites function of a browser, including:", + "p0032": "a detecting module, configured to detect a favorites selection operation of a user, where a favorites object selected by the favorites selection operation is at least partial displayed content on a tab currently displayed on the browser; a first response module, configured to store related data of the at least partial displayed content in response to the favorites selection operation, where the related data is used to display the at least partial displayed content; and a second response module, configured to display a favorites tab in response to a view operation of the user, where the at least partial displayed content is displayed on the favorites tab.", + "p0033": "According to the second aspect, in a possible implementation of the second aspect, the first response module stores the related data of the at least partial displayed content in response to the favorites selection operation in the following manner:", + "p0034": "in response to the favorites selection operation, determining the related data of the at least partial displayed content that is selected by the user to add to favorites; storing the related data of the at least partial displayed content as a resource package corresponding to the at least partial displayed content; determining a storage path of the resource package corresponding to the at least partial displayed content and a name of the resource package; and storing the storage path of the resource package corresponding to the at least partial displayed content and the name of the resource package.", + "p0035": "According to the second aspect, in a possible implementation of the second aspect, the apparatus further includes:", + "p0036": "a third response module, configured to: in response to the favorites selection operation, determine a basic parameter corresponding to the at least partial displayed content that is selected by the user to add to favorites, where the basic parameter is for describing a favorites type corresponding to the at least partial displayed content and at least one of location information and size information of the at least partial displayed content when the at least partial displayed content is displayed on the favorites tab; and store the basic parameter corresponding to the at least partial displayed content.", + "p0037": "According to the second aspect, in a possible implementation of the second aspect, the basic parameter includes a favorites type parameter corresponding to the at least partial displayed content; and the second response module displays the favorites tab in response to the view operation of the user in the following manner:", + "p0038": "in response to the view operation of the user, obtaining the stored storage path of the resource package corresponding to the at least partial displayed content, the stored name of the resource package, and the stored basic parameter corresponding to the partial displayed content; and when it is determined, based on the obtained basic parameter corresponding to the partial displayed content, the partial displayed content needs to be added to favorites and displayed, displaying the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content.", + "p0039": "According to the second aspect, in a possible implementation of the second aspect, the basic parameter further includes a location parameter and a size parameter that correspond to the at least partial displayed content; and the second response module displays the favorites tab based on the storage path of the resource package corresponding to the at least partial displayed content, the name of the resource package, and the basic parameter corresponding to the partial displayed content in the following manner:", + "p0040": "determining, based on the basic parameter corresponding to the partial displayed content, a first location and a first size of the at least partial displayed content on the favorites tab when the at least partial displayed content is displayed on the favorites tab; finding the related data of the at least partial displayed content based on the storage path of the resource package and the name of the resource package; and displaying the at least partial displayed content at the first location of the favorites tab based on the found related data of the at least partial displayed content, where a size of the partial displayed content displayed on the favorites tab is the first size.", + "p0041": "According to the second aspect, in a possible implementation of the second aspect, the apparatus further includes: a fourth response module, configured to: in response to a movement operation of the user, move the at least partial displayed content displayed on the favorites tab from the first location to a second location.", + "p0042": "According to the second aspect, in a possible implementation of the second aspect, the apparatus further includes: a fifth response module, configured to: in response to a scaling operation of the user, adjust the at least partial displayed content displayed on the favorites tab from the first size to a second size.", + "p0043": "According to the second aspect, in a possible implementation of the second aspect, the apparatus further includes: a sixth response module, configured to: in response to a hiding operation of the user, hide the at least partial displayed content displayed on the favorites tab.", + "p0044": "According to the second aspect, in a possible implementation of the second aspect, the related data of the at least partial displayed content includes: the at least partial displayed content or a uniform resource locator corresponding to the at least partial displayed content.", + "p0045": "According to the second aspect, in a possible implementation of the second aspect, the related data of the at least partial displayed content further includes:", + "p0046": "a display style file of the at least partial displayed content and/or a script file of the at least partial displayed content, where the display style file of the at least partial displayed content is for describing a display style of the at least partial displayed content on the currently displayed tab; and the script file of the at least partial displayed content is for describing operation logic of the at least partial displayed content on the currently displayed tab.", + "p0047": "According to the second aspect, in a possible implementation of the second aspect, the at least partial displayed content includes one or more of a video, an audio, a picture, or a text.", + "p0048": "According to a third aspect, this application provides a computer-readable storage medium. The computer-readable storage medium stores instructions, and when the instructions are executed on an electronic device, the electronic device is enabled to perform the method for implementing a favorites function in any one of the first aspect and the possible implementations of the first aspect.", + "p0049": "According to a fourth aspect, this application provides a computer program product. The computer program product includes instructions, and when the instructions are executed by one or more processors, the instructions are used to implement the method for implementing a favorites function in any one of the first aspect and the possible implementations of the first aspect.", + "p0050": "According to a fifth aspect, this application provides an electronic device, including:", + "p0051": "a memory, configured to store instructions; and one or more processors, where when the instructions are executed by the one or more processors, the processor performs the method for implementing a favorites function in any one of the first aspect and the possible implementations of the first aspect.", + "p0052": "BRIEF DESCRIPTION OF DRAWINGS", + "p0053": "FIG. 1A shows a schematic diagram of a display interface of a computer browser according to some embodiments of this application; FIG. 1B shows a schematic diagram of an interface of a browser favorites folder according to some embodiments of this application; FIG. 2A to FIG. 2I show schematic diagrams of some display interfaces of a browser according to some embodiments of this application; FIG. 3 shows a block diagram of a hardware structure of a computer according to some embodiments of this application; FIG. 4 shows a schematic diagram of a software architecture of a computer according to some embodiments of this application; FIG. 5 shows a schematic flowchart of a method for implementing a favorites function of a browser according to some embodiments of this application; FIG. 6A to FIG. 6C show schematic diagrams of some display interfaces of a browser according to some embodiments of this application; FIG. 7 shows a schematic flowchart of another method for implementing a favorites function of a browser according to some embodiments of this application; FIG. 8A and FIG. 8B show schematic diagrams of some display interfaces of a browser according to some embodiments of this application; FIG. 9 shows a schematic flowchart of another method for implementing a favorites function of a browser according to some embodiments of this application; FIG. 10A to FIG. 10D show schematic diagrams of some display interfaces of a browser according to some embodiments of this application; and FIG. 11 shows a schematic flowchart of adding content on two different tabs of a browser to a favorites tab for parallel display according to some embodiments of this application.", + "p0054": "DESCRIPTION OF EMBODIMENTS", + "p0055": "Illustrative embodiments of this application include but are not limited to one method for implementing a favorites function of a browser, a readable medium, and an electronic device.", + "p0056": "For ease of understanding the technical solutions of this application, the following first briefly describes technical terms in some embodiments of this application.", + "p0057": "A hypertext markup language is a standard markup language for creating a web page. The HTML is a basic technology that is often used to design a web page, a web application, and a user interface of a mobile application. A web browser may read an HTML file and render the HTML file into a visual web page.", + "p0058": "A uniform resource locator is commonly referred to as a web page address, which is referred to as a website address for short. The uniform resource locator is an address of a standard resource on the Internet, which is similar to a doorplate on the network.", + "p0059": "A cascading style sheet is a style sheet language for describing presentation of documents such as an HTML. The CSS describes how elements displayed on a screen should be rendered.", + "p0060": "Iframe is an HTML tag, and is used for a document in a document or a floating framework. The <iframe> tag specifies an inline framework. The inline framework is used to embed another document in a current HTML document.", + "p0061": "Grister is a front-end drag plug-in.", + "p0062": "JavaScript is a lightweight, interpreted, or just-in-time compiled programming language with function-first.", + "p0063": "To make the objectives, technical solutions, and advantages of this application clearer and more comprehensible, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that specific embodiments described herein are merely used to explain this application but are not intended to limit this application.", + "p0064": "For example, in the background, some existing browsers only add complete content of a web page selected by a user to favorites, and a layout manner of a display interface of a favorites folder is inappropriate. This leads to a technical problem that the user has poor experience when using favorites functions of some existing browsers.", + "p0065": "Therefore, an embodiment of this application provides a method for implementing a favorites function of a browser. Specifically, when an electronic device detects a favorites operation performed by the user on content of an area selected by the user on a tab currently displayed on the browser, in response to the favorites operation of the user, the electronic device stores, related data corresponding to the content of the area. The related data corresponding to the content of the area can be used to display the content of the area. In addition, the related data corresponding to the content of the area may include the content of the area , or a URL corresponding to the content of the area, a display style file for describing a specific style of displaying the content of the area, and a script file for describing operation logic about the content of the area of displaying the content of the area.", + "p0066": "In addition, when a view operation of the user is detected, for example, when an operation of switching, by the user, a current display page of the browser to a favorites tab of the browser by using the mouse is detected, the favorites tab of the browser is displayed, where the favorites tab displays the content that is selected by the user to add to favorites. The favorites tab is a tab that is of the browser and that is specially used to display a resource that is added to favorites.", + "p0067": "Therefore, an electronic device is enabled to add, based on a requirement of the user, partial resources in which the user is interested on the tab of the browser to favorites, to meet the requirement of the user for adding the partial resources on the tab of the browser to favorites. In addition, the favorites tab intuitively displays a plurality of resources that are added to favorites, to help the user quickly find resources in which the user is interested in the plurality of resources displayed on the favorites tab, so as to improve use experience when the user uses the favorites function of the browser.", + "p0068": "For example, in some embodiments, a plurality of pictures are displayed on a tab 01 of a browser installed on a computer 100 shown in FIG. 2A . After the user selects a landscape picture P1 by using a mouse and right-clicks the mouse, a favorites menu 121 pops up on the tab 01. After the user taps an \"add to favorites and currently display\" control 122 by using the mouse, the picture P1 is added to favorites on a favorites tab 02 shown in FIG. 2B . When the user switches, by using the mouse, a tab currently displayed on the browser from the tab 01 to the favorites tab, the computer 100 jumps to a user interface shown in FIG. 2B . Refer to FIG. 2B , the foregoing landscape picture P1 that is added to favorites is displayed, so that the user can intuitively view the picture P1 on the favorites tab 02.", + "p0069": "For another example, in some embodiments, a video vo1 is displayed on a tab 03 of the browser installed on the computer 100 shown in FIG. 2C . After the user selects the video vo1 by using the mouse and right-clicks the mouse, a favorites menu 121 pops up on the tab 03. After the user taps the \"add to favorites and currently display\" control 122 by using the mouse, the video vo1 is also added to the favorites tab 02 shown in FIG. 2D .", + "p0070": "It should be noted that, after the video vo1 is added to favorites on the favorites tab 02, the browser still currently displays the tab 03 on which the video vo1 is located, and the browser displays the favorites tab 02 shown in FIG. 2D only when the user manually switches a display interface of the browser to the favorites tab 02.", + "p0071": "In addition, the user may further customize a typesetting manner of the picture P1 and the video vo1 that are added to favorites on the favorites tab 02. For example, the user drags, by using the mouse, a control T in a lower right corner of the picture P1 or the video vo1 that is already added to favorites and displayed on the favorites tab 02 shown in FIG. 2D , to zoom in or zoom out a card area of the picture P1 or the video vo1. For another example, the user drags, by using the mouse, the picture P1 or the video vo1 that is added to favorites and displayed on the favorites tab 02 shown in FIG. 2D , to move the picture P1 or the video vo1 to another location on the favorites tab 02.", + "p0072": "In addition, in some cases, when the user does not want to display the picture P1 or the video vo1 that is added to favorites and displayed on the favorites tab 02, the user may alternatively tap a control S in an upper right corner of the picture P1 or the video vo1 shown in FIG. 2D , to hide the picture P1 or the video vo1.", + "p0073": "For example, the user taps the control S in the upper right corner of the picture P1 on the favorites tab 02 shown in FIG. 2D , and the favorites tab 02 shown in FIG. 2D jumps to a display status shown in FIG. 2E . Refer to FIG. 2E , the picture P1 is hidden, and only a name of the picture P1 is displayed in a favorites market on a left side of the favorites tab 02.", + "p0074": "It may be understood that FIG. 2A to FIG. 2E show examples of some user interfaces in which the computer 100 performs the method for implementing a favorites function of a browser provided in some embodiments of this application, to add partial resources selected by the user on the tab of the browser to favorites and display the resources on the favorites tab 02. In some other embodiments, by performing the method for implementing a favorites function of a browser provided in some embodiments of this application, the computer 100 can also add all content on some tabs selected by the user to favorites, and display, on the favorites tab 02 of the browser in a card form based on the requirement of the user, all content on a tab that is added to favorites.", + "p0075": "For example, in an embodiment shown in FIG. 2F , content currently displayed on a tab 04 is partial chapters of a novel nov1. The user taps a favorites control 114 on the tab 04 of the browser, and a favorites menu 121 pops up on the tab 04. After the user taps the \"add to favorites and currently display\" control 122 by using the mouse, the partial chapters of the novel nov1 displayed on the tab 04 are added to favorites on the favorites tab 02 shown in FIG. 2G , and the novel nov1, the video vo1, and the picture P1 are all intuitively displayed on the favorites tab 02. It should be noted that, after the partial chapters of the novel nov1 are added to favorites on the favorites tab 02, the browser still currently displays the tab 04 on which the partial chapters of the novel nov1 are located, and the browser displays the favorites tab 02 shown in FIG. 2G only when the user manually switches a display interface of the browser to the favorites tab 02. Similarly, the user may also perform operations such as zooming in, zooming out, and moving in a display interface of the novel nov1 displayed on the favorites tab 02.", + "p0076": "In addition, in some embodiments, the user may further simultaneously select a plurality of resources from the tab currently displayed on the browser to add to favorites. For example, in an embodiment shown in FIG. 2H , after the user selects the picture P1 and a picture P2 by using the mouse and right-clicks the mouse, the favorites menu 121 pops up on the tab 01. After the user taps the \"add to favorites and currently display\" control 122 by using the mouse, the picture P1 and the picture P2 are added to favorites on the favorites tab 02 shown in FIG. 2I . Similarly, the user may also perform operations such as zooming in, zooming out, and moving in a display interface of the picture P1 and the picture P2 displayed on the favorites tab 02.", + "p0077": "It should be noted that specific implementation processes involved in the foregoing embodiments are described in detail below with reference to specific flowcharts. Details are not described herein.", + "p0078": "It is not difficult to learn from the foregoing descriptions of embodiments shown in FIG. 2A to FIG. 2I that, by performing the method for implementing a favorites function of a browser provided in some embodiments of this application, the computer 100 may add partial resources selected by the user on a tab of the browser to favorites, to meet the requirement of the user for adding the partial resources on the tab to favorites. In addition, the computer 100 may further display a resource that is added to favorites by using a favorites tab specially used to display a favorites resource, so that the user can view and quickly find a favorites resource in which the user is interested. The user may further select, on the favorites tab, a favorites resource that does not need to be displayed, and customize a size, a location, and the like of an area occupied by each favorites resource displayed on the favorites tab, to improve use experience when the user uses the favorites function of the browser.", + "p0079": "In addition, in the foregoing embodiments, the browser installed on the computer 100 is used as an example to describe the technical solutions of this application. It may be understood that the technical solutions of this application are implemented by any electronic device that has a display screen and that can run an executable program of a browser, including but not limited to a mobile phone, a tablet computer, a laptop computer, a server, a wearable device, a head-mounted display, a mobile email device, a head unit device, a portable game console, a portable music player, a reader device, a television embedded in or coupled to one or more processors, or another electronic device that can access a network. For ease of description, the following continues to use the computer 100 as an example to describe the technical solutions of this application.", + "p0080": "With reference to FIG. 3 , the following describes in detail a hardware structure of the computer 100 in the foregoing embodiments. In FIG. 3 , similar components have a same reference numeral. As shown in FIG. 3 , the computer 100 may include a processor 110, a power module 140, a memory 180, a camera 170, a communication module 120, a sensor module 190, an audio module 150, an interface module 160, a display screen 102, and the like.", + "p0081": "It may be understood that the structure illustrated in this embodiment of this application constitutes no specific limitation on the computer 100. In some other embodiments of this application, the computer 100 may include more or fewer components than those shown in the figure, or some components may be combined, or some components may be split, or different component arrangements may be used. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.", + "p0082": "The processor 110 may include one or more processing units, for example, may include a processing module or a processing circuit of a central processing unit , a graphics processing unit , a digital signal processor , a micro-programmed control unit , an artificial intelligence processor, a programmable logic device , or the like. Different processing units may be independent components, or may be integrated into one or more processors.", + "p0083": "In some instances of this application, the processor 110 may be configured to detect whether a trigger event about adding partial resources or all resources on a function tab of a browser to favorites is generated. In addition, when it is detected that a trigger event about adding the partial resources on the function tab is generated, the resources to be added to favorites, a display style file corresponding to the resources, and a script file are stored locally. In addition, a storage path and a resource name that corresponds to the foregoing partial resources are stored, and a parameter such as a favorites type corresponding to the foregoing partial resources is stored.", + "p0084": "In some instances of this application, the processor 110 may be configured to: determine, based on a favorites type parameter of the partial resources, whether the partial resources need to be displayed on a favorites tab; when it is determined that the partial resources need to be displayed on the favorites tab, display the partial resources on the favorites tab based on the storage path and the resource name that are of the partial resources and that are stored in an information sharing unit 221, and the parameter such as coordinates and a size that corresponds to the partial resources and that is stored in a parameter storage unit 222; and when it is determined that the partial resources do not need to be displayed on the favorites tab, add the partial resources to favorites in a favorites market based on the storage path and the resource name that are of the partial resources and that are stored in the information sharing unit 221.", + "p0085": "The memory 180 may be configured to store data, a software program, and a module; and may be a volatile memory such as a random-access memory , may be a non-volatile memory , such as a read-only memory , a flash memory , a hard disk drive , or a solid-state drive , may be a combination of the foregoing types of memories, or may be a removable storage medium such as a secure digital storage card. Specifically, the memory 180 may include a program storage area and a data storage area . The program storage area may store program code, and the program code is used to enable the processor 110 to execute the program code to perform the method for implementing a favorites function provided in embodiments of this application. In some embodiments of this application, the data storage area may be configured to store partial resources to be added to favorites, display style files and script files of the partial resources. In some other embodiments of this application, the data storage area may be configured to store parameters such as coordinates, sizes, and favorites types of the partial resources.", + "p0086": "The power module 140 may include a power supply, a power management component, and the like. The power supply may be a battery. The power management component is configured to manage charging of the power supply and power supplying to another module performed by the power supply. The charging management module is configured to receive charging input from a charger. The power management module is configured to connect to the power supply, the charging management module, and the processor 110.", + "p0087": "The communication module 120 may include a mobile communication module and/or a wireless communication module. The mobile communication module may include but is not limited to an antenna, a power amplifier, a filter, a low noise amplifier , and the like. The wireless communication module may include an antenna, and receive and send an electromagnetic wave through the antenna. The wireless communication module may provide a wireless communication solution that is applied to the computer 100 and that includes a wireless local area network network), Bluetooth , a global navigation satellite system , frequency modulation , near field communication technology, an infrared technology, and the like. The computer 100 may communicate with a network and another device by using the communication module.", + "p0088": "The camera 170 is configured to capture a static image or a video. An optical image of an object is generated through a lens, and is projected onto a photosensitive element. The photosensitive element converts an optical signal into an electrical signal, and then transmits the electrical signal to an ISP to convert the electrical signal into a digital image signal. The computer 100 may implement an image shooting function by using the ISP, the camera 170, a video codec, a GPU , the display screen 102, an application processor, and the like.", + "p0089": "The display screen 102 includes a display panel. The display panel may be a liquid crystal display , an organic light-emitting diode , an active-matrix organic light-emitting diode and active-matrix organic light-emitting diode , a flexible light-emitting diode , a mini-LED, a micro LED, a micro OLED, a quantum dot light-emitting diode , or the like. For example, the display screen 102 is configured to display a user interface such as a function tab and a favorites tab of the browser.", + "p0090": "The sensor module 190 may include an optical proximity sensor, a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.", + "p0091": "The audio module 150 may convert digital audio information into an analog audio signal for output, or convert analog audio input into a digital audio signal. The audio module 150 may be further configured to encode and decode an audio signal. In some embodiments, the audio module 150 may be disposed in the processor 110, or some function modules of the audio module 150 are disposed in the processor 110. In some embodiments, the audio module 150 may include a speaker, an earpiece, a microphone, and a headset interface.", + "p0092": "The interface module 160 includes an external memory interface, a universal serial bus interface, a subscriber identity module card interface, and the like.", + "p0093": "It may be understood that the hardware structure shown in FIG. 3 constitutes no specific limitation on the computer 100. In some other embodiments of this application, the computer 100 may include more or fewer components than those shown in FIG. 3 , or some components may be combined, or some components may be split, or different component arrangements may be used.", + "p0094": "With reference to FIG. 4 , the following describes in detail a system architecture of the computer 100 in the foregoing embodiments. Specifically, as shown in FIG. 4 , a system of the computer 100 includes an application layer 210, a framework layer 220, a kernel layer 230, and a hardware layer 240.", + "p0095": "The application layer 210 may include a series of application packages, for example, Browser 211, and other applications such as Calendar and Map.", + "p0096": "The framework layer 220 includes an information sharing unit 221, a parameter storage unit 222, a display processing unit 223, an event detection unit 224, a global event management unit 225, and the like. The event detection unit 224 is configured to detect whether the browser generates a trigger event about adding partial resources or all resources on a function tab to favorites.", + "p0097": "The information sharing unit 221 is configured to store a path and a resource name of a resource that is selected by a user in the browser to add to favorites. In some embodiments, the information sharing unit 221 is further configured to store a favorites type of the resource that is selected by the user in the browser to add to favorites. In some embodiments, the information sharing unit 221 is a localstorage . The information sharing unit 221 may include a collectlist field and a collectstatus field. Content corresponding to the collectlist field is storage paths and resource names of a plurality of resources that have been added to favorites or are being added to favorites and that are stored in the information sharing unit 221. Content of the collectstatus field is in one-to-one correspondence with the content of the collectlist field, and the content of the collectstatus field is for describing a favorites type of a resource corresponding to the collectlist field. For example, if the content of collectstatus is current, it indicates that the favorites type of the corresponding resource is to be added to favorites and currently displayed. If the content of collectstatus is later, it indicates that the favorites type of the corresponding resource is not to be displayed after being added to favorites.", + "p0098": "The parameter storage unit 222 is configured to store a name and a favorites type of a favorites resource, and parameters such as coordinates, a size, and a name corresponding to the favorites resource when the favorites resource needs to be displayed on a favorites tab of the browser.", + "p0099": "The display processing unit 223 is configured to listen to the collectlist field in the information sharing unit 221 of the computer 100 in real time. The content corresponding to the collectlist field is the storage paths and the resource names of the plurality of resources that have been added to favorites or are being added to favorites and that are stored in the information sharing unit 221. When the display processing unit 223 of the computer 100 listens to that a storage path and a resource name are newly added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines that a new resource needs to be added to favorites. The display processing unit 223 determines, based on a favorites type parameter of the new resource, whether the resource needs to be displayed on the favorites tab.", + "p0100": "The global event management unit 225 is configured to create events for a sub-window and a top-layer top window of different iframe tags on the favorites tab of the browser, to implement communication between different iframe tags on the favorites tab, and implement mutual communication between the plurality of favorites resources on the favorites tab of the browser.", + "p0101": "In some other embodiments, the system of the computer 100 may further include a system library layer, and the system library layer is located between the framework layer and the kernel layer. This is not limited herein.", + "p0102": "The kernel layer 230 includes a display driver 231, configured to drive a display screen 102 to display a user interface such as a function tab and a favorites tab of the browser. In another embodiment, the kernel layer 230 may alternatively include another drive, for example, a touch control drive or an audio drive. This is not limited herein.", + "p0103": "The hardware layer 240 includes the display screen 102, and the display screen 102 is configured to display the user interface such as the function tab and the favorites tab of the browser. In another embodiment, the hardware layer 240 may alternatively include physical components such as a speaker and a motor. This is not limited herein.", + "p0104": "It can be learned from the foregoing descriptions of the schematic diagrams of the plurality of interfaces shown in FIG. 2A to FIG. 2G that, by performing the method for implementing a favorites function of a browser provided in embodiments of this application, the computer 100 may add partial resources or all resources on the function tab of the browser to favorites, and display, based on a requirement of the user, a resource that is added to favorites by the user on the favorites tab of the browser.", + "p0105": "With reference to the schematic diagrams of the plurality of interfaces shown in FIG. 2A to FIG. 2G and the system architecture diagram of the computer 100 shown in FIG. 4 , the following separately describes in detail a process of adding partial content on the function tab of the browser to favorites and adding all content on the function tab of the browser to favorites.", + "p0106": "First, a flowchart shown in FIG. 5 is used as an example to describe in detail a process of adding the partial content on the function tab of the browser to favorites provided in some embodiments of this application. The steps in the flowchart shown in FIG. 5 may be performed by the computer 100. Specifically, the flowchart shown in FIG. 5 includes the following steps.", + "p0107": "S501: Display the function tab of the browser.", + "p0108": "It should be understood that the computer 100 displays, by using the display screen 102, the function tab of the browser installed on the computer 100. The function tab of the browser may be a tab that is of the browser and that is used to display content searched by the user. For example, the function tab of the browser may be the tab 01 shown in FIG. 2A on which pictures such as flowers, characters, and landscape are displayed. For another example, the function tab of the browser may be the tab 03 shown in FIG. 2C on which the video vo1 is displayed.", + "p0109": "S502: When it is detected that a trigger event about adding the partial resources on the function tab to favorites is generated, locally store the partial resources to be added to favorites, a URL of the partial resources, and a display style file and a script file that correspond to the partial resources.", + "p0110": "For example, in some embodiments, when detecting a trigger event about adding the picture P1 shown in FIG. 2A to favorites, the event detection unit 224 of the computer 100 uses the picture P1 or a URL corresponding to the picture P1, and a display style file and a script file that correspond to the picture P1 as a resource package of the picture P1, and locally stores the resource package in the computer 100. A local storage path of the resource package of the picture P1 in the computer 100 is denoted as Path1, and a name of the resource package is denoted as Name1. In some embodiments, the name of the resource package may be a name of a resource of the picture: P1. It should be understood that the name of the resource package may be randomly set based on a requirement. This is not limited in this application.", + "p0111": "For another example, in some embodiments, when detecting a trigger event about adding the video vo1 shown in FIG. 2C to favorites, the event detection unit 224 of the computer 100 uses the video vo1 or a URL corresponding to the video vo1, and a display style file and a script file that correspond to the video vo1 as a resource package of the video vo1, and locally stores the resource package in the computer 100. A local storage path of the resource package of the video vo1 in the computer 100 is denoted as Path2, and a name of the resource package is denoted as Name2. In some embodiments, the name of the resource package may be a name of a resource of the video: vo1.", + "p0112": "In addition, in some embodiments, the user may further simultaneously select a plurality of resources from a tab currently displayed on the browser to add to favorites. For example, when detecting a trigger event about adding the picture P1 and the picture P2 shown in FIG. 2H to favorites, the event detection unit 224 of the computer 100 uses the picture P1 and the picture P2, or a URL of the picture P1 and a URL of the picture P2, and display style files and script files that correspond to the picture P1 and the picture P2 as a resource package of the picture P1 and the picture P2, and locally stores the resource package in the computer 100. A local storage path of the resource package of the picture P1 and the picture P2 in the computer 100 is denoted as Path0, and a name of the resource package is denoted as Name0. In some embodiments, the name of the resource package may be a name of a resource of the pictures: P1 & P2. It should be understood that the name of the resource package may be randomly set based on a requirement. This is not limited in this application.", + "p0113": "In addition, the display style file corresponding to the partial resources is for describing a display style of a display element included in the partial resources. For example, if the partial resources are a piece of text selected by the user, the display element included in the partial resources may be a font, a size, a color, a background color of the text, and a special format of the text. In some embodiments, the display style file corresponding to the partial resources may be specifically a CSS code file obtained by using a crawler technology from a tab corresponding to the partial resources.", + "p0114": "In addition, the script file corresponding to the partial resources is for describing running logic corresponding to the partial resources, for example, the running logic is that after the partial resources are selected on the function tab by using a mouse, a prompt box pops up, or a prompt box pops up after the partial resources are selected and right-clicked by using the mouse. In some embodiments, the script file corresponding to the partial resources may be specifically a JavaScript script file obtained by using a crawler technology from a tab corresponding to the partial resources.", + "p0115": "In some embodiments, the trigger event about adding the partial resources on the function tab to favorites may be, for example, that the user selects the partial resources on the tab by using the mouse, and taps a control related to favorites. For example, as shown in FIG. 2A , the user selects the landscape picture P1 on the tab 01 by using the mouse and right-clicks the mouse, and the favorites menu 121 pops up. After the user taps the \"add to favorites and currently display\" control by using the mouse, the browser generates the foregoing trigger event about adding the partial resources on the function tab to favorites.", + "p0116": "S503: Store a storage path and a resource name that correspond to the partial resources in the information sharing unit 221, and store a basic parameter corresponding to the partial resources in the parameter storage unit 222.", + "p0117": "The basic parameter of the partial resources includes the name and the favorites type of the partial resources, and parameters such as coordinates, a size, and a name that correspond to the partial resources when the partial resources are displayed on the favorites tab of the browser. In some embodiments, the favorites type of the partial resources includes: being added to favorites and currently displayed, and not being displayed after being added to favorites.", + "p0118": "In some embodiments, the information sharing unit 221 is a localstorage , and is configured to store a storage path and a resource name of a resource that is selected by the user in the browser to add to favorites. In some embodiments, the information sharing unit 221 is further configured to store a favorites type of the resource that is selected by the user in the browser to add to favorites.", + "p0119": "It should be understood that the basic parameter such as the storage path, the resource name, and the favorites type of the resource that is selected by the user to add to favorites are stored in different storage space, so that resource favorites and display do not affect each other, to help improve stability of the browser.", + "p0120": "For example, in some embodiments, the storage path Path1 corresponding to the picture P1 and the name P1 of the picture P1 are stored in the information sharing unit 221; and the name P1 of the picture P1 and a favorites type are stored in the parameter storage unit 222. For another example, the storage path Path2 corresponding to the video vo1 and the name vo1 of the video vo1 are stored in the information sharing unit 221; and the name vo1 of the video vo1 and a favorites type are stored in the parameter storage unit 222. For another example, in some embodiments, the storage path Path0 corresponding to the picture P1 and the picture P2 and the name P1 & P2 of the picture P1 and the picture P2 is stored in the information sharing unit 221; and the name P1 & P2 of the picture P1 and the picture P2 and a favorites type are stored in the parameter storage unit 222.", + "p0121": "In some embodiments, coordinates and sizes that correspond to favorites resources and that are stored in the parameter storage unit 222 is preset fixed parameters. In some embodiments, coordinates and sizes that correspond to favorites resource and that are stored in the parameter storage unit 222 may alternatively be parameters determined in real time based on a current typesetting status of the favorites tab.", + "p0122": "S504: Determine, based on the basic parameter of the partial resources, whether the partial resources need to be displayed on the favorites tab. For example, it may be determined, based on a favorites type parameter included in the basic parameter of the partial resources, whether the partial resources need to be displayed on the favorites tab. If the favorites type is to be added to favorites and currently displayed, it indicates that the partial resources need to be displayed on the favorites tab, and S505 is performed. If the favorites type is not to be displayed after being added to favorites, it indicates that the partial resources do not need to be displayed on the favorites tab, and the partial resources may be added to a favorites market of the browser, and S506 is performed.", + "p0123": "For example, in some embodiments, the display processing unit 223 of the computer 100 listens to the collectlist field in the information sharing unit 221 of the computer 100 in real time. The content corresponding to the collectlist field is the storage paths and the resource names of the plurality of resources that have been added to favorites or are being added to favorites and that are stored in the information sharing unit 221. When the display processing unit 223 of the computer 100 listens to that a storage path and a resource name are newly added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines that a new resource needs to be added to favorites. The display processing unit 223 determines, based on a favorites type parameter of the new resource, whether the resource needs to be displayed on the favorites tab.", + "p0124": "In some embodiments, when a storage path and a resource name of a new resource are written into the information sharing unit 221, a favorites type parameter corresponding to the resource may also be written into the information sharing unit 221, for example, written into the collectstatus field in the information sharing unit 221, so that the display processing unit 223 determines, based on the content of the collectstatus field, whether to display the new resource on the favorites tab of the browser. For example, when a value of the collectstatus field is current, the display processing unit 223 determines that partial resources on the function tab need to be displayed on the favorites tab; or when the value of the collectstatus field is later, the display processing unit 223 determines that the partial resources on the function tab do not need to be displayed on the favorites tab.", + "p0125": "S505: Display the partial resources on the favorites tab based on the storage path and the resource name of the partial resources that are stored in the information sharing unit 221 and the basic parameter that corresponds to the partial resources and that is stored in the parameter storage unit 222.", + "p0126": "For example, in some embodiments, when the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the picture P1 are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that the favorites type of the picture P1 is: to be added to favorites and currently displayed. Therefore, based on parameters such as coordinates and a size of the picture P1 stored in the parameter storage unit 222, an iframe tag A is inserted into a favorites tab 02 shown in FIG. 6A , and the storage path and the resource name of the picture P1 are written into src in the iframe, to obtain the picture P1, and the display style file and the script file that correspond to the picture P1, so that the picture P1 is displayed on the tab 02 shown in FIG. 2B based on the display style described by the display style file corresponding to the picture P1, and the running logic described by the script file. In some embodiments, code of the iframe tag A shown in FIG. 6A may be, for example,<frameset rows=\"xx,*\" frameborder='xx'><frame src=\"Path1\"name=\"P1\"><frameset cols=\"xx\"><frameset rows=\"xx\">", + "p0127": "Similarly, in some embodiments, when the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the video vo1 on the tab 03 shown in FIG. 2C are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that the favorites type of the video vo1 is: to be added to favorites and currently displayed. Therefore, based on parameters such as coordinates and a size of the video vo1 stored in the parameter storage unit 222, an iframe tag B is inserted into the favorites tab 02 shown in FIG. 6A , and the storage path and the resource name of the video vo1 are written into src in the iframe, to obtain the video vo1, and the display style file and the script file that correspond to the video vo1, so that the video vo1 is displayed on the tab 02 shown in FIG. 2D based on the display style described by the display style file corresponding to the video vo1, and the running logic described by the script file. In some embodiments, code of the iframe tag B shown in FIG. 6A may be, for example,<frameset rows=\"xx,*\" frameborder='xx'><frame src=\"Path2\"name=\"vo1\"><frameset cols=\"xx\"><frameset rows=\"xx\">", + "p0128": "It should be noted that, when the browser completes adding the video vo1 to favorites and displays the video vo1 on the tab 02 shown in FIG. 2D , interface content of the video vo1 displayed on the tab 02 is consistent with interface content of the video vo1 to be added to favorites on the tab 03 shown in FIG. 2C . For example, when the user selects the video vo1 on the tab 03 by using the mouse to add the video vo1 to favorites, a playback progress of the video vo1 is one minute. After the browser completes adding the video vo1 to favorites, when the user opens the favorites tab 02, the video vo1 displayed on the favorites tab 02 is also a pause interface of playing the video vo1 to one minute.", + "p0129": "In some embodiments, the user may further adjust locations of the picture P1 and the video vo1 that are displayed on the favorites tab 02, and perform operations such as zooming in and zooming out on the picture P1 and the video vo1. For example, a Grister plug-in is installed in the browser, and the user separately selects and drags the control T in the lower right corner of the picture P1 and the video vo1 by using the mouse, to implement a zoom-in operation on the picture P1 and the video vo1.", + "p0130": "In addition, in some other embodiments, the user may further select some resources from the plurality of favorites resources displayed on the favorites tab 02, and add the resources to a favorites market of the browser. For example, the user taps the control S in the upper right corner of the picture P1 shown in FIG. 2D , and the favorites tab 02 of the browser enters the user interface shown in FIG. 2E . The picture P1 is no longer displayed on the favorites tab 02, but is added to the favorites market on the left side of the favorites tab 02.", + "p0131": "In this way, the user can customize, based on a requirement of the user, a resource that needs to be displayed on the favorites tab of the browser, and adjust locations and sizes of resources displayed on the favorites tab of the browser. This can greatly improve use experience when the user uses the favorites function of the browser.", + "p0132": "S506: Add the partial resources to favorites in the favorites market based on the storage path and the resource name of the partial resources that are stored in the information sharing unit 221.", + "p0133": "For example, in some embodiments, the user selects a picture P3 on a tab 01 shown in FIG. 6B by using the mouse, then right-clicks the mouse, a favorites menu 121 pops up on the tab 01, and the user taps a \"not displayed after being added to favorites\" control 123 by using the mouse. After detecting an operation of tapping the control 123 by the user, the event detection unit 224 uses the picture P3, and a display style file and a script file that correspond to the picture P3 as a resource package of the picture P3, and locally stores the resource package in the computer 100. A local storage path of the resource package of the picture P3 in the computer 100 is denoted as Path3, and a name of the resource package is denoted as Name3. In some embodiments, the name of the resource package may be a name of a resource of the picture: P3. When the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the picture P3 are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that a favorites type of the picture P3 is: not being displayed after being added to favorites, and adds the picture P3 to a favorites market shown in FIG. 6C . After the user taps the picture P3 in the favorites market, the display processing unit 223 may be triggered to obtain the storage path of the picture P3, and the picture P3 is displayed on the favorites tab 02. In this way, a requirement that the user customizes whether the resource added to favorites is intuitively displayed on the favorites tab 02 can be implemented, to help improve use experience when the user uses the favorites function of the browser.", + "p0134": "After the process of adding the partial resources on the function tab of the browser to favorites provided in some embodiments of this application is described, the following describes in detail a process of adding all resources in the function tab of the browser to favorites provided in some other embodiments of this application.", + "p0135": "The following uses a flowchart shown in FIG. 7 as an example to describe in detail the process of adding all the content on the function tab of the browser to favorites provided in some embodiments of this application. The steps in the flowchart shown in FIG. 7 may be performed by the computer 100. Specifically, the flowchart shown in FIG. 7 includes the following steps.", + "p0136": "S701: Display the function tab of the browser.", + "p0137": "For example, in the embodiment shown in FIG. 2F , the computer 100 displays an interface of the tab 04 of the browser, and the content on the tab 04 is the partial chapters of the novel nov1.", + "p0138": "S702: When it is detected that a trigger event about adding all the resources on the function tab to favorites is generated, locally store all the resources on the tab, and display style files and script files that correspond to all the resources on the tab.", + "p0139": "In some embodiments, the trigger event about adding all the resources on the function tab to favorites is: The user taps a control related to favorites on the function tab. For example, as shown in FIG. 2F , the user taps the favorites control 114 on the tab 04, and the favorites menu 121 pops up. After the user taps the \"add to favorites and currently display\" control 122 by using the mouse, the browser generates the foregoing trigger event about adding all the resources on the function tab to favorites.", + "p0140": "In some other embodiments, the trigger event about adding all the resources on the function tab to favorites may alternatively be: The user selects all the content on the function tab, and taps a control related to favorites on the function tab. For example, the user selects all the content on the tab 04 shown in FIG. 2F by using the mouse and right-clicks the mouse, and the favorites menu 121 pops up. Then, after the user taps the \"add to favorites and currently display\" control 122 by using the mouse, the browser generates the foregoing trigger event about adding all the resources on the function tab.", + "p0141": "It should be understood that the trigger event about adding all the resources on the function tab to favorites in the foregoing embodiments is merely some example descriptions for ease of understanding of the technical solutions of this application. During actual application, how to specifically generate the trigger event may be determined based on an actual situation. This is not limited in this application.", + "p0142": "For example, in some embodiments, all the content displayed on the tab 04 shown in FIG. 2F is the partial chapters of the novel nov1. When detecting a trigger event about adding all the content on the tab 04 shown in FIG. 2F to favorites, the event detection unit 224 of the computer 100 uses the novel nov1, and a display style file and a script file that correspond to the novel nov1 that are displayed on the tab 04 as a resource package of the novel nov1, and locally stores the resource package in the computer 100. A local storage path of the resource package of the novel nov1 in the computer 100 is denoted as Path4, and a name of the resource package is denoted as Name4. In some embodiments, the name of the resource package may be a name of the novel nov1: nov1. It should be understood that the name of the resource package may be randomly set based on a requirement. This is not limited in this application. For meanings of the display style files and the script files corresponding to all the resources on the tab, refer to the related text descriptions of S502 in FIG. 5 . Details are not described herein again.", + "p0143": "S703: Store storage paths and resource names corresponding to all the resources on the tab in the information sharing unit 221, and store basic parameters corresponding to all the resources in the parameter storage unit 222.", + "p0144": "The basic parameters of all the resources include the names and favorites types of all the resources, and parameters such as coordinates, sizes, and names corresponding to all the resources when all the resources are displayed on the favorites tab of the browser. In some embodiments, the favorites types of all the resources include: being added to favorites and currently displayed, and not being displayed after being added to favorites.", + "p0145": "For example, in some embodiments, the storage path Path4 corresponding to the novel nov1 on the tab 04 and the name nov1 of the novel nov1 are stored in the information sharing unit 221. The name nov1 of the novel nov1 and a favorites type are stored in the parameter storage unit 222.", + "p0146": "In addition, for descriptions of the information sharing unit 221 and the parameter storage unit 222, refer to the related text descriptions of S503 in FIG. 5 . Details are not described herein again.", + "p0147": "S704: Determine, based on the basic parameters of all the resources on the tab, whether all the resources on the function tab need to be displayed on the favorites tab. For example, it may be determined, based on a favorites type parameter included in the basic parameters of all the resources, whether all the resources need to be displayed on the favorites tab. If the favorites type is to be added to favorites and currently displayed, it indicates that all the resources on the function tab need to be displayed on the favorites tab, and S705 is performed. If the favorites type is not to be displayed after being added to favorites, it indicates that all the resources on the function tab do not need to be displayed on the favorites tab, and S706 is performed.", + "p0148": "For example, in some embodiments, the display processing unit 223 of the computer 100 listens to the collectlist field in the information sharing unit 221 of the computer 100 in real time. When the display processing unit 223 of the computer 100 listens to that a storage path and a resource name are newly added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines that a new resource needs to be added to favorites. The display processing unit 223 determines, based on a favorites type parameter of the new resource, whether the resource needs to be displayed on the favorites tab. In some embodiments, the display processing unit 223 of the computer 100 may determine, based on a value of the collectstatus field in the information sharing unit 221, whether all the resources on the function tab need to be displayed on the favorites tab. For example, when a value of the collectstatus field is current, the display processing unit 223 determines that all the resources on the function tab need to be displayed on the favorites tab; or when the value of the collectstatus field is later, the display processing unit 223 determines that all the resources on the function tab do not need to be displayed on the favorites tab.", + "p0149": "S705: Display all the resources on the tab on the favorites tab based on the storage paths and the resource names of all the resources that are stored in the information sharing unit 221 and the basic parameters that correspond to all the resources on the tab and that are stored in the parameter storage unit 222.", + "p0150": "For example, in some embodiments, when the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the novel nov1 are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that the favorites type of the novel nov1 is: to be added to favorites and currently displayed. Therefore, based on parameters such as coordinates and a size of the novel nov1 stored in the parameter storage unit 222, an iframe tag C is inserted into the favorites tab 02 shown in FIG. 6A , and the storage path and the resource name of the novel nov1 are written into src in the iframe, to obtain the novel nov1, and the display style file and the script file that correspond to the novel nov1, so that the novel nov1 is displayed on the favorites tab 02 shown in FIG. 2G based on the display style described by the display style file corresponding to the novel nov1, and the running logic described by the script file. In some embodiments, code of the iframe tag C shown in FIG. 6A may be, for example,<frameset rows=\"xx,*\" frameborder='xx'><frame src=\"Path4\"name=\"nov1\"><frameset cols=\"xx\"><frameset rows=\"xx\">", + "p0151": "Similarly, in some embodiments, the user may further adjust a location of the novel nov1 displayed on the favorites tab 02 shown in FIG. 2G , and perform operations such as zooming in and zooming out on the novel nov1. For example, a Grister plug-in is installed in the browser, and the user separately selects and drags the control T in the lower right corner of the picture P1 and the video vo1 by using the mouse, to implement a zoom-in operation on the picture P1 and the video vo1.", + "p0152": "For example, the user taps the control S in the upper right corner of the novel nov1 shown in FIG. 2G , and the favorites tab 02 of the browser enters a user interface shown in FIG. 8A . The novel nov1 is no longer displayed on the favorites tab 02, but is added to a favorites market on a left side of the favorites tab 02. In this way, the user can customize, based on a requirement of the user, a resource that needs to be displayed on the favorites tab of the browser, and adjust locations and sizes of resources displayed on the favorites tab of the browser. This can greatly improve use experience when the user uses the favorites function of the browser.", + "p0153": "S706: Add all the resources on the tab to favorites in the favorites market based on the storage paths and the resource names of all the resources on the tab that are stored in the information sharing unit 221.", + "p0154": "For example, in some embodiments, the user taps a favorites control 114 on a tab 04 shown in FIG. 8B , and a favorites menu 121 pops up on the tab 04. The user taps a \"not displayed after being added to favorites\" control 123 by using the mouse. After detecting an operation of tapping the control 123 by the user, the event detection unit 224 uses the novel nov1, and a display style file and a script file that correspond to the novel nov1 as a resource package of the novel nov1, and locally stores the resource package in the computer 100. A local storage path of the resource package of the novel nov1 in the computer 100 is denoted as Path5, and a name of the resource package is denoted as Name5 In some embodiments, the name of the resource package may be a name of a resource of the novel nov1: nov1. When the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the novel nov1 are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that a favorites type of the novel nov1 is: not being displayed after being added to favorites, and adds the novel nov1 to the favorites market shown in FIG. 8A . After the user taps the novel nov1 in the favorites market, the display processing unit 223 may be triggered to obtain the storage path of the novel nov1, and the novel nov1 is displayed on the favorites tab 02. In this way, a requirement that the user customizes whether the resource added to favorites is intuitively displayed on the favorites tab 02 can be implemented, to help improve use experience when the user uses the favorites function of the browser.", + "p0155": "The following still uses an example in which all the content on the tab 04 in the browser shown in FIG. 2F is the partial chapters of the novel nov1 to describe in detail, with reference to a flowchart shown in FIG. 9 , a process of adding all content of a function tab of the browser provided in some other embodiments of this application. The steps in the flowchart shown in FIG. 9 may be performed by the computer 100. Specifically, the flowchart shown in FIG. 9 includes the following steps.", + "p0156": "S901: Display the function tab of the browser.", + "p0157": "For example, in the embodiment shown in FIG. 2F , the computer 10 displays an interface of the tab 04 of the browser, and the content on the tab 04 is the partial chapters of the novel nov1.", + "p0158": "S902: When it is detected that a trigger event about adding all the resources on the function tab to favorites is generated, store a URL of the tab in the information sharing unit 221, and store basic parameters corresponding to all resources in the parameter storage unit 222.", + "p0159": "The basic parameters of all the resources include names and favorites types of all the resources, and parameters such as coordinates, sizes, and names corresponding to all the resources when all the resources are displayed on the favorites tab of the browser. In some embodiments, the favorites types of all the resources include: being added to favorites and currently displayed, and not being displayed after being added to favorites.", + "p0160": "In S902, in comparison with S702 shown in FIG. 7 , a difference only lies in that: in S902, when it is detected that the trigger event about adding all the resources on the function tab is generated, the URL corresponding to the tab to be added to favorites is stored in the information sharing unit 221 as a resource address. However, in S702, when it is detected that the trigger event about adding all the resources on the function tab to favorites is generated, all the content on the tab to be added to favorites is cached locally, and then a cached address and names of all the content on the tab added to favorites are stored in the information sharing unit 221 as resource addresses.", + "p0161": "For example, in some embodiments, the content displayed on the tab 04 shown in FIG. 2F is the partial chapters of the novel nov1. When detecting the trigger event about adding all the content on the tab 04 shown in FIG. 2F to favorites, the event detection unit 224 of the computer 100 locally stores the URL, for example, url2, of the tab 04 in the computer 100. In addition, a favorites type of the tab 04 is stored in the information sharing unit 221. For definitions of the information sharing unit 221 and the parameter storage unit 222, refer to the related text descriptions of S502 in FIG. 5 . Details are not described herein again.", + "p0162": "S903: Determine, based on the basic parameters of all the resources on the tab, whether all the resources on the tab need to be displayed on the favorites tab. For example, it may be determined, based on a favorites type parameter included in the basic parameters of all the resources, whether all the resources need to be displayed on the favorites tab. If the favorites type is to be added to favorites and currently displayed, it indicates that all the resources on the function tab need to be displayed on the favorites tab, and S904 is performed. If the favorites type is not to be displayed after being added to favorites, it indicates that all the resources on the function tab do not need to be displayed on the favorites tab, and S905 is performed. For details, refer to the related text descriptions of S704 in FIG. 7 . Details are not described herein again.", + "p0163": "S904: Display all the resources on the tab on the favorites tab based on the URL that is stored in the information sharing unit 221 and the basic parameters that correspond to all the resources on the tab and that are stored in the parameter storage unit 222.", + "p0164": "For example, in some embodiments, when the display processing unit 223 of the computer 100 listens to that the storage path and the resource name of the novel nov1 are added to the collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using the collectstatus field in the information sharing unit 221, that the favorites type of the novel nov1 is: to be added to favorites and currently displayed. Therefore, based on parameters such as coordinates and a size of the novel nov1 stored in the parameter storage unit 222, an iframe tag C is inserted into the favorites tab 02 shown in FIG. 6A , and the URL of the tab 04 corresponding to the novel nov1 are written into src in the iframe tag C, to obtain the novel nov1, the display style file corresponding to the novel nov1, and the script file, so that the display style is described based on the display style file corresponding to the novel nov1, the running logic is described based on the script file, and the novel nov1 is displayed on the favorites tab 02 shown in FIG. 2G .", + "p0165": "In some embodiments, code of the iframe tag C shown in FIG. 6A may be, for example,<frameset rows=\"xx,*\" frameborder='xx'><frame src=\"url2\"name=\"nov1\"><frameset cols=\"xx\"><frameset rows=\"xx\">", + "p0166": "Similarly, in some embodiments, the user may further adjust a location of the novel nov1 displayed on the favorites tab 02 shown in FIG. 2G , and perform operations such as zooming in and zooming out on the novel nov1. For details, refer to the related text descriptions of S705 in FIG. 7 . Details are not described herein again.", + "p0167": "S905: Add the URL corresponding to the tab to favorites in a favorites market.", + "p0168": "For example, the URL of the tab 04 corresponding to the novel nov1 is added to favorites in the favorites market shown in FIG. 8A . After the user taps the novel nov1 in the favorites market, the display processing unit 223 may be triggered to obtain the storage path of the novel nov1, and the novel nov1 is displayed on the favorites tab 02. In this way, a requirement that the user customizes whether the resource added to favorites is intuitively displayed on the favorites tab 02 can be implemented, to help improve use experience when the user uses the favorites function of the browser.", + "p0169": "With reference to the foregoing descriptions of embodiments in FIG. 5 to FIG. 9 , the processes of adding the partial resources or all the resources on the function tab of the browser to favorites that can be implemented in the technical solutions of this application is described in detail. To better understand the technical solutions of this application, the following further describes the technical solutions of this application with reference to a specific application scenario.", + "p0170": "Specifically, with reference to schematic diagrams of user interfaces shown in FIG. 10A to FIG. 10D and a flowchart shown in FIG. 11 , the following describes the technical solutions of this application by using a scenario in which a user uses a browser to concurrently read a plurality of resources as an example.", + "p0171": "A browser shown in FIG. 10A opens two function tabs and a favorites tab 02. An active tab displayed in FIG. 10A is the tab 04, and content displayed on the tab 04 is partial chapters of a novel nov1. A browser shown in FIG. 10B also opens two function tabs and the favorites tab 02, an active tab displayed in FIG. 10B is the tab 03, and content displayed on the tab 03 is a video vo1. FIG. 10C shows a schematic diagram of an iframe tag B and an iframe tag C that are in the favorites tab 02 and that respectively correspond to the tab 03 and the tab 04. A browser shown in FIG. 10D also opens two function tabs and the favorites tab 02, an active tab displayed in FIG. 10D is the favorites tab 02, and currently displayed content is content on the tab 03 and the tab 04, in other words, the content on the tab 03 and the content on the tab 04 are displayed in parallel on the favorites tab 02. According to some embodiments of this application, FIG. 11 shows a flowchart in which a computer 100 displays the content displayed on the tab 03 and the tab 04 of the browser in parallel on the favorites tab 02 by performing a method for implementing a favorites function of a browser provided in this application. Specifically, refer to FIG. 11 , a flowchart in which the computer 100 displays the content displayed on the tab 03 and the tab 04 of the browser in parallel on the favorites tab 02 includes the following steps.", + "p0172": "S1101: Detect that a trigger event about adding all content on the tab 03 and the tab 04 of the browser to favorites is generated.", + "p0173": "For example, in some embodiments, if the user wants to display the content on both the tab 03 and the tab 04 on the favorites tab 02, the user may tap a favorites control 114 on the current active tab 04, and tap an \"add to favorites and currently display\" control 122 on a pop-up favorites menu 121, so that the browser generates the trigger event about adding all the content on the tab 04 to favorites. Similarly, the user may tap the tab 03 shown in FIG. 10A by using a mouse, and the browser jumps to the user interface shown in FIG. 10B . The tab 03 becomes a current active tab of the browser. Then, the user may tap a favorites control 114 on the active tab 03, and tap an \"add to favorites and currently display\" control 122 in a pop-up favorites menu 121, so that the browser generates a trigger event about adding all the content on the tab 03 to favorites.", + "p0174": "S1102: Store both a URL of the tab 03 and a URL of the tab 04 in the information sharing unit 221, and store basic parameters corresponding to the tab 03 and the tab 04 in the parameter storage unit 222. The basic parameters of the tab 03 and the tab 04 are respectively names and favorites types of the tab 03 and the tab 04, and parameters such as coordinates, sizes, and names corresponding to the tab 03 and the tab 04 when the tab 03 and the tab 04 are displayed on the favorites tab of the browser.", + "p0175": "For example, in some embodiments, the URL of the tab 04 is url2, and the URL of the tab 03 is url3. When detecting the trigger event correspond to the tab 03 and the tab 04 is generated in S1101, the event detection unit 224 of the computer 100 stores both url2 and url3 in the information sharing unit 221 of the computer 100, and stores the parameters such as the favorites types of the tab 03 and the tab 04 in the parameter storage unit 222. It should be understood that, because the user selects the \"add to favorites and currently display\" on both the tab 03 and the tab 04, the favorites types corresponding to both the tab 03 and the tab 04 are to be added to favorites and currently displayed. In addition, the parameter storage unit 222 may further store the parameters such as the resource names, the coordinates, and the sizes corresponding to the tab 03 and the tab 04, where the parameters of the coordinates and the sizes may be fixed parameters. For example, provided that the trigger event is detected, a corresponding resource to be added to favorites, a favorites type, a resource name and parameters such as default coordinates and a default size are written into the parameter storage unit 222.", + "p0176": "S1103: When it is determined that resources corresponding to the tab 03 and the tab 04 need to be displayed on the favorites tab 02, display the content on both the tab 03 and the tab 04 on the favorites tab 02 based on the URL of the tab 03, the URL of the tab 04, and the basic parameters that correspond to the tab 03 and the tab 04 and that are stored in the parameter storage unit 222.", + "p0177": "For example, in some embodiments, when the display processing unit 223 of the computer 100 listens to that URLs respectively corresponding to the tab 03 and the tab 04 are added to a collectlist field in the information sharing unit 221, the display processing unit 223 determines, by using a collectstatus field in the information sharing unit 221, that the favorites types of the tab 03 and the tab 04 are: to be added to favorites and currently displayed. In this case, based on the parameters, that are stored in the parameter storage unit 222 such as the coordinates and the sizes of the tab 03 and the tab 04, the iframe tag B corresponding to the tab 03 and the iframe tag C corresponding to the tab 04 are respectively inserted into the favorites tab 02 shown in FIG. 10C . Then, the URL of the tab 03 and the URL of the tab 04 are respectively written into src of the iframe tag B and src of the iframe tag C, so that the content on the tab 03 and the tab 04 are displayed in parallel on the favorites tab 02 shown in FIG. 10D .", + "p0178": "Similarly, in some embodiments, the user may further adjust locations of the tab 03 and the tab 04 that are displayed on the favorites tab 02 shown in FIG. 10D , and perform operations such as zooming in and zooming out on the tab 03 and the tab 04. For details, refer to the related text descriptions of S705 in FIG. 7 . Details are not described herein again.", + "p0179": "In addition, in some embodiments, the content on the tab 03 and the tab 04 that are displayed in parallel on the favorites tab 02 may be further interacted. For example, it is assumed that a video played on the tab 03 is a TV series that is photographed after being adapted based on a novel on the tab 04, events for a sub-window and a top-layer top window of the iframe tag B and the iframe tag C may be created by using the global event management unit 225 of the computer 100. In this way, communication between the iframe tag B and the iframe tag C is implemented, to implement synchronous display of the video vo1 and the novel nov1 shown in FIG. 10D .", + "p0180": "In addition, an embodiment of this application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, steps in the foregoing method embodiments can be implemented.", + "p0181": "An embodiment of this application provides a computer program product. When the computer program product runs on an electronic device, the electronic device is enabled to implement the steps in the foregoing method embodiments when executing the computer program product.", + "p0182": "An embodiment of this application further provides an electronic device. The electronic device includes: at least one processor, a memory, and a computer program that is stored in the memory and that may be run on the at least one processor. When executing the computer program, the processor implements the steps in any one of the foregoing method embodiments.", + "p0183": "Embodiments of a mechanism disclosed in this application may be implemented in hardware, software, firmware, or a combination of these implementations. Embodiments of this application may be implemented as a computer program or program code executed in a programmable system. The programmable system includes at least one processor, a storage system , at least one input device, and at least one output device.", + "p0184": "The program code may be applied to input instructions to perform the functions described in this application and generate output information. The output information may be applied to one or more output devices in a known manner. For purposes of this application, a processing system includes any system having a processor such as a digital signal processor , a micro-controller, an application-specific integrated circuit , or a microprocessor.", + "p0185": "The program code may be implemented in a high-level programming language or an object-oriented programming language to communicate with the processing system. The program code can also be implemented in an assembly language or a machine language when needed. The mechanisms described in this application are not limited to the scope of any particular programming language. In either case, the language may be a compiled language or an interpreted language.", + "p0186": "In some cases, disclosed embodiments may be implemented by hardware, firmware, software, or any combination thereof. Disclosed embodiments may be implemented as instructions carried by or stored thereon by one or more temporary or non-transitory machine-readable storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed through a network or another computer-readable medium. Therefore, the machine-readable medium may include any mechanism for storing or transmitting information in a machine -readable form. The machine-readable medium includes but is not limited to a floppy disk, an compact disc, an optical disc, a read-only memory , a magneto-optical disc, a read-only memory , a random access memory , an erasable programmable read-only memory , an electrically erasable programmable read-only memory , a magnetic or an optical card, a flash memory, or a tangible machine-readable memory that is configured to transmit information through Internet by using electricity, light, sound or another form of propagating signal . Therefore, the machine-readable medium includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a form that is readable by a machine .", + "p0187": "In the drawings, some structural or method features may be shown in a particular arrangement and/or sequence. However, it should be understood that such a specific arrangement and/or sequence may not be required. Rather, in some embodiments, the features may be arranged in a manner and/or order other than that shown in the illustrative drawings. In addition, the inclusion of structural or method features in a particular figure is not meant to imply that such features are required in all embodiments, and in some embodiments, these features may not be included or may be combined with other features.", + "p0188": "It should be noted that: All units/modules mentioned in the device embodiments of this application are logical units/modules. Physically, one logical unit/module may be one physical unit/module, or may be a part of one physical unit/module, or may be implemented through a combination of a plurality of physical units/modules. Physical implementations of these logical units/modules are not the most important, and a combination of functions implemented by these logical units/modules is a key to resolving the technical problem raised in this application. In addition, to highlight an innovative part of this application, the foregoing device embodiments of this application do not introduce a unit/module that is not closely related to resolving the technical problem proposed in this application. This does not indicate that no other unit/module exists in the foregoing device embodiments.", + "p0189": "It should be noted that: In the examples and specification of this patent, relational terms such as first and second are merely used to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In addition, the terms \"include\" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, a method, an article, or a device that includes a list of elements not only includes those elements but also includes other elements that are not expressly listed, or further includes elements inherent to such a process, method, article, or device. An element preceded by \"includes a ...\" does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or device that includes the element.", + "p0190": "Although this application has been illustrated and described with reference to some embodiments of this application, it should be understood by a person of ordinary skill in the art that various changes may be made thereto in form and detail without departing from the scope of this application.", + "features": { + "F1": "A method for implementing a favorites function of a browser, applied to an electronic device on which the browser is installed.", + "F2": "The method comprises detecting a favorites selection operation of a user, where a favorites object selected is at least partial displayed content on a tab currently displayed on the browser.", + "F3": "Storing related data of the at least partial displayed content in response to the favorites selection operation, where the related data is used to display the at least partial displayed content.", + "F4": "Displaying a favorites tab in response to a view operation of the user, with the at least partial displayed content displayed on the favorites tab.", + "F5": "Storing related data in response to the favorites selection operation by determining the related data of the at least partial displayed content, storing it as a resource package, determining a storage path and name of the resource package, and storing these.", + "F6": "Determining a basic parameter corresponding to the at least partial displayed content that is selected by the user to add to favorites, with the basic parameter describing a favorites type and at least one of location or size information.", + "F7": "The basic parameter comprises a favorites type parameter, and displaying the favorites tab involves obtaining the storage path and name of the resource package, and the basic parameter, and using these to display the tab.", + "F8": "The basic parameter further includes a location parameter and a size parameter, used to determine a first location and size for displaying the at least partial displayed content on the favorites tab.", + "F9": "In response to a movement operation of the user, moving the at least partial displayed content displayed on the favorites tab from the first location to a second location.", + "F10": "In response to a scaling operation of the user, adjusting the at least partial displayed content displayed on the favorites tab from the first size to a second size.", + "F11": "In response to a hiding operation of the user, hiding the at least partial displayed content displayed on the favorites tab.", + "F12": "The related data of the at least partial displayed content comprises the content itself or a uniform resource locator corresponding to it.", + "F13": "The related data further includes a display style file and/or a script file, where the display style file describes a display style and the script file describes operation logic.", + "F14": "The at least partial displayed content comprises one or more of a video, an audio, a picture, and a text.", + "F15": "An apparatus for implementing a favorites function of a browser, comprising a detecting module to detect a favorites selection operation and a first response module to store related data.", + "F16": "A second response module is configured to display a favorites tab in response to a view operation of the user.", + "F17": "The first response module stores related data by determining and storing it as a resource package with a storage path and name.", + "F18": "A third response module determines a basic parameter, which includes a favorites type and location or size information, and stores it.", + "F19": "The second response module displays the favorites tab using the storage path, resource package name, and basic parameter.", + "F20": "The fourth response module moves the at least partial displayed content from a first location to a second on the favorites tab in response to user movement.", + "F21": "The fifth response module adjusts the size of the content displayed on the favorites tab in response to user scaling.", + "F22": "The sixth response module hides the at least partial displayed content on the favorites tab in response to user hiding operations.", + "F23": "A computer-readable storage medium stores instructions to enable an electronic device to perform the method for implementing a favorites function.", + "F24": "A computer program product includes instructions to implement the method for implementing a favorites function when executed by one or more processors.", + "F25": "An electronic device comprises a memory to store instructions and one or more processors to perform the method for implementing a favorites function." + } + } + }, + { + "FAN": "76353708", + "CPC": [ + "G06F-016/00", + "G06F-016/221", + "G06F-016/2237", + "G06F-016/2246", + "G06F-016/2255", + "G06F-016/2272", + "G06F-016/319", + "G06F-017/30/315", + "G06F-017/30/324", + "G06F-017/30/327", + "G06F-017/30/33", + "G06F-017/30/336", + "G06F-017/30/622" + ], + "Content": { + "title": "Index establishment method and device", + "pa01": "Embodiments of the present invention provide an index establishment method and device. The method can include determining, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column; determining an index type according to the data information of the column; and establishing the index for the column according to the index type based on the determination that the index is to be established for the column.", + "c-en-0001": "An index establishment method, comprising: determining, according to index status information of a column in a database within a time threshold, whether an index is to be established for the column, wherein determining whether the index is to be established for the column comprises: acquiring the index status information of the column within the time threshold, determining, according to the index status information, whether a current index is used for the column within the time threshold, in response to the current index being used for the column within the time threshold, determining whether a number of times that the current index has been used within the time threshold is not less than a preset number-of-times threshold, and determining that an index is to be established for the column, in response to the number of times the current index has been used being not less than the number-of-times threshold; determining an index type of a plurality of index types according to the data information of the column, wherein the plurality of index types comprise a B-Tree index and a Hash index and the determined index type is: the B-tree index when the column is a continuous value type, and the Hash index when a Join occurs in the column; and establishing an index for the column according to the index type based on a determination that an index is to be established for the column.", + "c-en-0002": "The method of claim 1 , further comprising: in response to the determination that an index is not to be established for the column, determining, after the time threshold, whether an index is to be established for the column.", + "c-en-0003": "The method of claim 2 , wherein determining, according to the index status information of the column in the database within the preset time threshold, whether an index is to be established for the column further comprises: determining that an index is not to be established for the column, in response to the current index not being used for the column within the time threshold or the number of times that the current index has been used being less than the number-of-times threshold.", + "c-en-0004": "The method of claim 1 , wherein the plurality of index types further comprise an inverted index and a Bitmap index, and the determined index type is: the inverted index in response to the number of words in the column being greater than a preset number-of-words threshold; and the bitmap index in response to the number of words in the column being not greater than a preset number-of-words threshold, a Join not occurring, and the column being a discontinuous value type.", + "c-en-0005": "The method of claim 1 , further comprising: receiving a retrieval expression; dividing the retrieval expression into a plurality of sub-expressions; querying whether a retrieval result corresponding to a sub-expression of the plurality of sub-expressions exists in a cache; and in response to the retrieval result existing, generating a retrieval response to be returned to the user according to the retrieval result corresponding to the sub-expression and retrieval results of other sub-expressions; or in response to a retrieval result corresponding to the sub-expression not existing, performing retrieval on the sub-expression by using an index of the column, obtaining a retrieval result corresponding to the sub-expression, and storing the obtained retrieval result in the cache after a retrieval response is generated according to the obtained retrieval result and retrieval results of other sub-expressions.", + "c-en-0006": "The method of claim 1 , further comprising, before determining, according to the index status information of the column in the database within the preset time threshold, whether an index is to be established for the column: initializing the database; constructing a first set of indexes for each column in the database according to a default index type; and reconstructing a second set of indexes for each column after a preset period of time.", + "c-en-0007": "An index establishment device, comprising: a memory storing a set of instructions, and one or more processors configured to execute the set of instructions to cause the device to perform: determining, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column, wherein determining whether the index is to be established for the column comprises: acquiring the index status information of the column within the time threshold, determining, according to the index status information, whether a current index is used for the column within the time threshold, in response to the current index being used for the column within the time threshold, determining whether a number of times that the current index has been used within the time threshold is not less than a preset number-of-times threshold, and determining that an index is to be established for the column, in response to the number of times that the current index has been used being not less than the number-of-times threshold; and determining an index type of a plurality of index types according to the data information of the column, wherein the plurality of index types comprise a B-Tree index and a Hash index and the determined index type is: the B-tree index when the column is a continuous value type, and the Hash index when a Join occurs in the column; and establishing an index for the column according to the index type based on a determination that an index is to be established for the column.", + "c-en-0008": "The device of claim 7 , wherein the one or more processors are further configured to execute the set of instructions to cause the device to perform: in response to the determination that an index is not to be established for the column, determining, after the time threshold, whether an index is to be established for the column.", + "c-en-0009": "The device of claim 8 , wherein the one or more processors are further configured to execute the set of instructions to cause the device to perform: determining that an index is not to be established for the column in response to the current index not being used for the column within the time threshold or the number of times the current index has been used being less than the number-of-times threshold.", + "c-en-0010": "The device of claim 8 , wherein the plurality of index types further comprise an inverted index and a Bitmap index, and the determined index type is: the inverted index in response to the number of words in the column being greater than a preset number-of-words threshold; or the bitmap index in response to the number of words in the column being not greater than a preset number-of-words threshold, a Join not occurring, and the column being a discontinuous value type.", + "c-en-0011": "The device of claim 8 , wherein the one or more processors are further configured to execute the set of instructions to cause the device to perform: receiving a retrieval expression and divide the retrieval expression into a plurality of sub-expressions; querying whether a retrieval result corresponding to a sub-expression of the plurality of sub-expressions exists in a cache; generating, according to the retrieval results of the sub-expressions, a retrieval response to be returned to the user in response to the retrieval result existing, or performing retrieval on the sub-expression by using an index of the column, obtaining a retrieval result corresponding to the sub-expression, and storing the obtained retrieval result in the cache after a retrieval response is generated according to the obtained retrieval result and retrieval results of other sub-expressions, in response to a retrieval result corresponding to the sub-expression not existing.", + "c-en-0012": "The device of claim 8 , wherein the one or more processors are further configured to execute the set of instructions to cause the device to perform: initializing the database; constructing a first set of indexes for each column in the database according to a default index type; and reconstructing a second set of indexes for each column after a preset period of time.", + "c-en-0013": "A non-transitory computer readable medium that stores a set of instructions that is executable by at least one processor of a computing system to cause the computing system to perform an index establishment method, the method comprising: determining, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column wherein determining whether the index is to be established for the column comprises: acquiring the index status information of the column within the time threshold, determining, according to the index status information, whether a current index is used for the column within the time threshold, in response to the current index being used for the column within the time threshold, determining whether a number of times that the current index has been used within the time threshold is not less than a preset number-of-times threshold, and determining that an index is to be established for the column, in response to the number of times the current index has been used being not less than the number-of-times threshold; determining an index type of a plurality of index types according to the data information of the column, wherein the plurality of index types comprise a B-Tree index and a Hash index and the determined index type is: the B-tree index when the column is a continuous value type, and the Hash index in response when a Join occurs in the column; and establishing an index for the column according to the index type based on a determination that an index is to be established for the column.", + "c-en-0014": "The non-transitory computer readable medium of claim 13 , wherein the method further comprises: in response to the determination that an index is not to be established for the column, determining, after the time threshold, whether an index is to be established for the column.", + "c-en-0015": "The non-transitory computer readable medium of claim 14 , wherein determining, according to the index status information of the column in the database within the preset time threshold, whether an index is to be established for the column further comprises: determining that an index is not to be established for the column, in response to the current index not being used for the column within the time threshold or the number of times the current index has been used being less than the number-of-times threshold.", + "c-en-0016": "The non-transitory computer readable medium of claim 13 , wherein the plurality of index types further comprise an inverted index and a Bitmap index, and the determined index type is: the inverted index in response to the number of words in the column being greater than a preset number-of-words threshold; and the bitmap index in response to the number of words in the column being not greater than a preset number-of-words threshold, a Join not occurring, and the column being a discontinuous value type.", + "c-en-0017": "The non-transitory computer readable medium of claim 13 , further comprising: receiving a retrieval expression; dividing the retrieval expression into a plurality of sub-expressions; querying whether a retrieval result corresponding to a sub-expression of the plurality of sub-expressions exists in a cache; and in response to the retrieval result existing, generating, according to the retrieval results of the sub-expressions, a retrieval response to be returned to the user, or in response to a retrieval result corresponding to the sub-expression not existing, performing retrieval on the sub-expression by using an index of the column, obtaining a retrieval result corresponding to the sub-expression, and storing the obtained retrieval results in the cache after a retrieval response is generated according to the obtained retrieval results of the sub-expressions.", + "c-en-0018": "The non-transitory computer readable medium of claim 13 , further comprising, before determining, according to the index status information of the column in the database within the preset time threshold, whether an index is to be established for the column: initializing the database; constructing a first set of indexes for each column in the database according to a default index type; and reconstructing a second set of indexes for each column after a preset period of time.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATION", + "p0002": "The disclosure claims the benefits of priority to International application number PCT/CN2016/106581, filed Nov. 21, 2016, and Chinese application number 201510868254.X, filed Dec. 1, 2015, both of which are incorporated herein by reference in their entireties.", + "p0003": "BACKGROUND", + "p0004": "An index is a structure for sorting values in one or more columns in a database table. Information in the database table can be rapidly accessed by using an index. Indexes provide pointers pointing to data values stored in columns of the table. These pointers are then sorted according to a sorting order specified by a user. An index can be searched to find a specific value, and a row including the value can be determined according to a pointer.", + "p0005": "With the continuous development of Internet technology, a massive amount of data is generated from people's daily activities on the Internet. How to rapidly find required data from the stored massive amount of data becomes a major concern for technical personnel. A user can specify an index type for an index in a conventional database. However, for a same index type, different types of data can have different lengths of query time. As a result, if an index type is inappropriately set, a user may spend an excessively long period of time to make a query in a database, resulting in a poor user experience.", + "p0006": "Conventionally, existing index types are limited and lead to poor query performance in some scenarios.", + "p0007": "Index types of conventional data are limited and are dominated by B-Tree indexes. B-Tree indexes are not suitable for all data features. For example, search speeds are very slow for the Key-Value mode. In Structured Query Language with joins, multiple searches will be performed for one SQL. The performance of SQL with joins can be severely affected if a B-Tree is used for searching.", + "p0008": "Furthermore, conventionally, logs are manually analyzed to optimize indexes, resulting in poor operability and maintainability.", + "p0009": "Indexes bring very high maintenance costs for a conventional database having a large amount of data. Consequently, user access logs can be viewed every night to decide which columns require new indexes, which columns do not need indexes, and which columns are to be combined to establish a combined index. Such automatic adjustment of index types by using history statistics ) is completely manual, resulting in very poor operability and maintainability.", + "p0010": "Therefore, how to combine the usage of a database to establish suitable indexes for columns in the database to improve data retrieval efficiency and save operation and maintenance resources becomes a technical problem.", + "p0011": "SUMMARY OF THE DISCLOSURE", + "p0012": "Embodiments of the disclosure provide an index establishment method. The method can include: determining, according to index status information of a column in a database within a time threshold, whether an index is to be established for the column; determining an index type according to the data information of the column; and establishing an index for the column according to the index type based on a determination that an index is to be established for the column.", + "p0013": "In some embodiments, the method further includes: in response to the determination that an index is not to be established for the column, determining, after the time threshold, whether an index is to be established for the column.", + "p0014": "In some embodiments, determining, according to the index status information of the column in the database within the time threshold, whether an index is to be established for the column further comprises: acquiring the index status information of the column within the time threshold; determining, according to the index status information, whether an index is used for the column within the time threshold; and in response to an index being used for the column within the time threshold, determining whether a number of times that an index for the column has been used within the time threshold is not less than a preset number-of-times threshold; and determining that an index is not to be established for the column in response to an index not being used for the column within the time threshold or the number of times being less than the number-of-times threshold; and deter mining that an index is to be established for the column in response to the number of times being not less than the number-of-times threshold.", + "p0015": "In some embodiments, the index types at least include a B-Tree index, a Hash index, an inverted index, and a Bitmap index, and determining the index type according to the data information of the column further comprises: determining that the index type is the B-Tree index in response to the column being a continuous value type; determining that the index type is the Hash index in response to a Join occurring in the column; and determining that the index type is the inverted index in response to the number of words in the column being greater than a preset number-of-words threshold; or determining that the index type is the bitmap index in response to the number of words in the column being not greater than a preset number-of-words threshold, a Join not occurring, and the column being a discontinuous value type.", + "p0016": "In some embodiments, the method further includes: receiving a retrieval expression; dividing the retrieval expression into a plurality of sub-expressions; querying whether a retrieval result corresponding to a sub-expression of the plurality of sub-expressions exists in a cache; and in response to the retrieval result existing, generating, a retrieval response to be returned to the user according to the retrieval result corresponding to the sub-expression and retrieval results of other sub-expressions; or in response to a retrieval result corresponding to the sub-expression not existing, performing retrieval on the sub-expression by using an index of the column, obtaining a retrieval result corresponding to the sub-expression, and storing the obtained retrieval result in the cache after a retrieval response is generated according to the obtained retrieval result and retrieval results of other sub-expressions.", + "p0017": "In some embodiments, before determining, according to the index status information of the column in the database within the preset time threshold, whether an index is to be established for the column, the method further includes: initializing the database; constructing a first set of indexes for each column in the database according to a default index type; and reconstructing a second set of indexes for each column after a preset period of time.", + "p0018": "Embodiments of the present application further provide an index establishment device. The device can include: a determination module configured to determine, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column; and an establishment module configured to determine an index type according to the data information of the column, and establish an index for the column according to the index type based on a determination that an index is to be established for the column.", + "p0019": "In some embodiments, the establishment module is further configured to, in response to the determination that an index is not to be established for the column, determine, after the time threshold, whether an index is to be established for the column.", + "p0020": "In some embodiments, the determination module is further configured to: acquire the index status information of the column within the time threshold; determine, according to the index status information, whether an index is used for the column within the time threshold; in response to an index being used for the column within the time threshold, determine whether a number of times that an index has been used within the time threshold is not less than a preset number-of-times threshold; determine that an index is not to be established for the column in response to an index not being used for the column within the time threshold or the number of times being less than the number-of-times threshold; and determine that an index is be established for the column, in response to the number of times being not less than the number-of-times threshold.", + "p0021": "In some embodiments, the index types at least comprise a B-Tree index, a Hash index, an inverted index, and a Bitmap index, and the establishment module is further configured to: determine that the index type is the B-Tree index in response to the column being a continuous value type; determine that the index type is the Hash index in response to a Join occurring in the column; and determines that the index type is the inverted index in response to the number of words in the column being greater than a preset number-of-words threshold; or determines that the index type is the bitmap index in response to the number of words in the column being not greater than a preset number-of-words threshold, a Join not occurring, and the column being a discontinuous value type.", + "p0022": "In some embodiments, the device can further include: a division module configured to receive a retrieval expression and divide the retrieval expression into a plurality of sub-expressions; a query module configured to query whether a retrieval result corresponding to a sub-expression of the plurality of sub-expressions exists in a cache; a processing module configured to generate, according to the retrieval results of the sub-expressions, a retrieval response to be returned to the user in response to the retrieval result existing, or perform retrieval on the sub-expression by using an index of the column, obtain a retrieval result corresponding to the sub-expression, and store the obtained retrieval result in the cache after a retrieval response is generated according to the obtained retrieval result and retrieval results of other sub-expressions, in response to a retrieval result corresponding to the sub-expression not existing.", + "p0023": "In some embodiments, the device can further include an initialization module configured to initialize the database; construct a first set of indexes for each column in the database according to a default index type; and reconstruct a second set of indexes for each column after a preset period of time.", + "p0024": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0025": "FIG. 1 is a flowchart of an index establishment method, according to embodiments of the present application.", + "p0026": "FIG. 2 illustrates a schematic diagram of a data structure, according to embodiments of the present application.", + "p0027": "FIG. 3 is a flowchart of streamed result merging, according to embodiments of the present application.", + "p0028": "FIG. 4 illustrates a schematic structural diagram of an index, according to embodiments of the present application.", + "p0029": "FIG. 5 is a flowchart of establishing an index, according to embodiments of the present application.", + "p0030": "FIG. 6 illustrates a schematic structural diagram of an index establishment device, according to embodiments of the present application.", + "p0031": "DETAILED DESCRIPTION", + "p0032": "Embodiments of the disclosure can determine, according to the index use status of each column in a database, whether an index is to be established for the column, and a suitable index type is selected based on the data information of the column during the establishment of an index. Therefore, the retrieval efficiency can be improved under the premise of saving hardware resources and reducing labor input.", + "p0033": "FIG. 1 is a flowchart of an index establishment method, according to embodiments of the present application. The method includes the following steps S 101 and S 102 .", + "p0034": "In step S 101 , it is determined, according to index status information of a column in a database within a preset time threshold, whether an index is to be established for the column.", + "p0035": "If it is determined that an index is to be established for the column, step S 102 can be performed", + "p0036": "The method can return to step S 101 , if it is determined that an index is not to be established for the column.", + "p0037": "If the database includes a massive amount of data, it is time consuming, labor consuming, and unnecessary to establish an index for each column in a database as a user does not often retrieve or query all columns. Therefore, in the present application, index status information can be introduced for each column in a database. It can be determined, based on the index status information within a period of time, whether an index needs to be established for the column. Meanwhile, to ensure that the index status information of each column can be adjusted according to the retrieval manners and habits of the user, the index status information of each column can be updated every time threshold. The index status information can include, for example, whether the column has used an index within the time threshold and whether a number of times that the index has been used reaches a particular number. In some embodiments, the time threshold can be a preset time interval value, representing a statistics-collection time range for collecting statistics on the index use status of a column. The time threshold may be adjusted according to a capacity of the database and a number of users. The time threshold may be set to a day, and may be extended or shortened.", + "p0038": "In some embodiments of the present application, step S 101 can further include: step a): acquiring the index status information of the column within the time threshold, and step b): determining, according to the index status information, whether a current index is used for the column within the time threshold. When the current index is used for the column within the time threshold, it can be determined whether the number of times that the current index has been used within the time threshold is not less than a preset number-of-times threshold.", + "p0039": "Therefore, it can be determined that an index is not to be established for the column if the current index is not used for the column within the time threshold or the number of times that the current index has been used for the column within the time threshold is less than the number-of-times threshold. It can also be determined that an index is to be established for the column if the number of times the current index for the column has been used within the time threshold is not less than the number-of-times threshold.", + "p0040": "In addition, none of the columns in the database has an established index after the initialization of a database. Therefore, an index can be constructed for each column in the database according to a default index type, and an index can be constructed for each column again when a preset moment is reached. Indexes that are uniformly established may be preset.", + "p0041": "In step S 102 , an index type can be determined according to the data information of the column, and an index can be established for the column according to the index type.", + "p0042": "An index type can be determined according to the data information of the column after it is determined in step S 101 that an index is to be established for the column. Therefore, based on the determined index type, personalized retrieval services can be provided to users based on different cases, so that the average time of querying the database by all users can be reduced.", + "p0043": "If it is determined that an index is not to be established for the column, after the time interval corresponding to a current time threshold ends, it is determined again, according to the index use status information of the column within the time interval corresponding to the next time threshold, whether an index needs to be established for the column. That is, step S 101 can be performed again. Therefore, flexible adjustment can be made according to the actual case.", + "p0044": "In some embodiments, index types can include Inverted index, B-Tree index, Hash index, and Bitmap index. Accordingly, the index type can be determined as a B-Tree index if the column is a continuous value type, as a Hash index if a Join occurs in the column, as an inverted index if the number of words in the column is greater than a preset number-of-words threshold, as a bitmap index if the number of words in the column is not greater than a preset number-of-words threshold, a Join does not occur, and the column is a discontinuous value type.", + "p0045": "It is appreciated that Join is an important operations in a relational database system. Common Joins in an SQL Server include an inner join, an outer join, a cross join, and the like. If data of a row in one table matching with a row in another table is being acquired from two or more tables, the Join operation may be used, as the Join is characterized by joining tables or functions to perform a query.", + "p0046": "For the inverted index, the number of words in the column can be the number of variable values included in the column.", + "p0047": "In some embodiments of the present application, if data in a column is a value type and is continuous , a range query can be performed on a field having the value type. As shown in a data structure of FIG. 2 , an intermediate node in the data structure of a B-Tree inherently has a range attribute. Therefore, for a column of a continuous value type, the efficiency of a range search on a B-Tree index can be higher than that of a range search on an inverted index and a bitmap index.", + "p0048": "In addition, it is assumed that there are n records in an entire index slice, field A has x different term values, and the compression ratio of an original compression algorithm of \u201cdelta+vint\u201d is p, a maximum value of p can be \u00bc. In this case, a result may be obtained by using an equation \u201cx<128\u00d7p\u201d transformed from \u201cx\u00d7n/32<n\u00d74\u00d7p\u201d.", + "p0049": "For example, when p=\u00bc, x<128\u00d7\u00bc=32 may be obtained.", + "p0050": "As can be seen from the foregoing result, if a number of words is less than 32, the space occupied when a bitmap index is used is smaller than that occupied when an inverted index is used. Therefore, the number-of-words threshold may be set to 32. However, it is appreciated that the number-of-words threshold may be subsequently changed based on another algorithm.", + "p0051": "Conventionally, after one user query is made, no result information related to the query can be saved. Generally, a query expression of a user is usually formed of a plurality of different sub-expressions. Therefore, if no atomic sub-expression is cached in a database, sub-expression query results cannot be reused for different SQLs having a same expression. As a result, when no atomic sub-expression is cached, the performance of Boolean operations in any dimensions in online analytical processing is poor. Therefore, in some embodiments of the disclosure, a retrieval expression sent by a user can be divided into a plurality of sub-expressions when the retrieval expression is received, and it is queried whether a retrieval result corresponding to each sub-expression exists in a cache.", + "p0052": "According to a retrieval result corresponding to the sub-expression and retrieval results of other sub-expressions if the retrieval result exists, a retrieval response can be generated to return the retrieval result to the user.", + "p0053": "Retrieval can be performed on the sub-expression by using an index of the column if a retrieval result corresponding to the sub-expression does not exist, and the retrieval result can be stored in the cache after a retrieval response to be returned to the user is generated according to the retrieval result and retrieval results of other sub-expressions.", + "p0054": "FIG. 3 illustrates a flowchart of streamed result merging, according to embodiments of the disclosure. An inverted link obtained through decompression after an index is queried can be stored in the memory by using a bitmap structure. In a streamed merging framework, Boolean operations between expressions can be obtained one by one through merging under the premise of not generating any intermediate objects. The entire streamed merging process can include four steps as below.", + "p0055": "In a first step, a queue creation component can generate an object tree of a queue engine , according to a \u201cwhere\u201d expression.", + "p0056": "In a second step, a selector tree of a row identification elector can be constructed according to the object tree of the Engine Query.", + "p0057": "In a third step, a row ID set obtained from different indexes can be inserted on a leaf node, where an intermediate node is a logic operation merger.", + "p0058": "In a fourth step, election can be performed on root nodes one by one, and a row ID can be collected.", + "p0059": "The above example is based on a case in which different SQL statements may have a same \u201cwhere\u201d sub-expression. Therefore, an atomic sub-expression can be cached to accelerate index query performance.", + "p0060": "According to embodiments of the disclosure, based on an index structure that is independent on a column level, the most suitable index type can be automatically selected according to costs and expenses, and history statistics can be used to automatically adjust the index type, and an atomic sub-expression can be cached at a same time. Therefore, index query performance can be accelerated while storage costs are reduced.", + "p0061": "FIG. 4 illustrates a schematic structural diagram of an index, according to embodiments of the present application. In some embodiments, the column-level index architecture shown in FIG. 4 can be autonomously implemented based on the principles and characteristics of indexes, such as an inverted index, bitmap index, Hash index, and B-Tree index. All four index types can be supported. The index types can be transparent to a user and may not be specified externally by the user. Instead, an index type can be automatically selected according to data features. Moreover, the multiple index types can use a same data structure to perform Boolean operations. An index from which a \u201cwhere\u201d sub-expression result is found may not be perceived on an engine layer. In addition, an index may be automatically optimized according to history statistics information without manual involvement.", + "p0062": "In some embodiments, the structural diagram of an index mainly can include: a streamed merger, an index manager, and a sub-expression cache.", + "p0063": "The streamed merger can unify query results of different indexes and interaction between computation layers. Results from queries using different indexes can be stored by using a bitmap. A streamed merge tree can then be generated according to Boolean operations of a \u201cwhere\u201d expression, and the streamed merger then can output row numbers that satisfy a \u201cwhere\u201d condition, one by one.", + "p0064": "The index manager is responsible for index management, type selection, and automatic index optimization procedures.", + "p0065": "The sub-expression cache can cache the \u201cwhere\u201d sub-expressions. Therefore, an index may not be queried when different SQLs have a same sub-expression, and a query may be directly made in the cache.", + "p0066": "Based on the foregoing description, in a schematic flowchart of establishing an index as shown in FIG. 5 , an index can be first established for each column in a database , and an index can be established again on a periodic basis and history SQL statistics data can be acquired . It is determined, based on the history data, whether an index is used for the column on the current day and whether the number of times that an index has been used exceeds a threshold . A different index type can be selected according to the type of data in the column after it is determined that an index is to be established for the column . Therefore, not only does it support the use of different index types for fields in different columns, but also an index type can be selected without being perceived by a user, so that storage costs can be saved and query speed can be increased.", + "p0067": "Embodiments of the present application further provide an index establishment device. As shown in FIG. 6 , the device includes: a determination module 610 and an establishment module 620 .", + "p0068": "Determination module 610 can determine, according to the index status information of a column in a database within a preset time threshold, whether an index is to be established for the column; and", + "p0069": "Establishment module 620 can determine an index type according to the data information of the column and establish an index for the column according to the index type when the determination module determines that an index is to be established for the column.", + "p0070": "In some embodiments, the establishment module can further determine, after the time threshold and according to index use status information of the column within the time threshold, whether an index is to be established for the column when the determination module determines that an index is not to be established for the column.", + "p0071": "In some embodiments, determination module 610 can further acquire the index status information of the column within the time threshold; determine, according to the index status information, whether a current index is used for the column within the time threshold, and when the current index is used for the column within the time threshold, determine whether the number of times the current index for the column is used within the time threshold is not less than a preset number-of-times threshold; and determine that an index is not being established for the column if the current index is not used for the column within the time threshold or the number of times an index for the column is used within the time threshold is less than the number-of-times threshold; or determine that an index needs to be established for the column if the number of times the current index for the column has been used within the time threshold is not less than the number-of-times threshold.", + "p0072": "In some embodiments, the index types at least include a B-Tree index, a Hash index, and a Bitmap index, and the establishment module can determine that the index type is a B-Tree index if the column is a continuous value type; determine that the index type is a Hash index if a join occurs in the column; and determines that the index type is an inverted index if the number of words in the column is greater than a preset number-of-words threshold; or determine that the index type is a bitmap index if the number of words in the column is not greater than a preset number-of-words threshold, a Join does not occur, and the column is a discontinuous value type.", + "p0073": "In some embodiments, the device further includes a division module, a query module, and a processing module. The division module can be configured to divide a retrieval expression sent by a user into a plurality of sub-expressions when the retrieval expression is received. The query module can be configured to query whether a retrieval result corresponding to each sub-expression exists in a cache. The processing module can be configured to generate, according to a retrieval result corresponding to the sub-expression and retrieval results of other sub-expressions, a retrieval response to be returned to the user when the retrieval result exists, or perform retrieval on the sub-expression by using an index of the column when a retrieval result corresponding to the sub-expression does not exist, and store the retrieval result in the cache after a retrieval response to be returned to the user is generated according to the retrieval result and retrieval results of other sub-expressions.", + "p0074": "In some embodiments, the device further includes an initialization module that can be configured to construct an index for each column in the database according to a default index type after the database has been initialized and to construct an index for each column again when a preset moment is reached.", + "p0075": "According to embodiments of the disclosure, an index can be dynamically established for each column in a database, and a suitable index type can be selected according to the actual case, so that data retrieval efficiency can be effectively improved under the premise of reducing resource consumption and labor input.", + "p0076": "It is appreciated that the present disclosure may be implemented by using software or software plus a hardware platform. Based on such an understanding, the technical solutions of the present invention may be implemented in the form of a software product. The software product may be stored in a non-volatile and/or non-transitory storage medium , and includes several instructions for instructing a computer device to perform the methods in the implementation scenarios of the present invention.", + "p0077": "It is appreciated that the accompanying drawings are only schematic diagrams of a preferred implementation scenario. The modules or procedures in the accompanying drawings are not necessarily indispensable for implementing the present invention.", + "p0078": "It is appreciated that the modules in the apparatus in the implementation scenario may be distributed in the apparatus in the implementation scenario according to the description of the implementation scenario, or may be correspondingly changed to be located in one or more apparatuses different from the apparatus in the implementation scenario. The modules in the foregoing implementation scenario may be combined into one module or may further be divided into a plurality of submodules.", + "p0079": "The foregoing sequence numbers in the present invention are merely for the convenience of description, and do not imply preference among the implementation scenarios.", + "p0080": "The foregoing disclosure is only several specific implementation scenarios of the present invention, but the present invention is not limited thereto. Any change conceivable by a person skilled in the art shall fall within the protection scope of the present invention.", + "features": { + "F1": "Determining, according to index status information of a column in a database within a time threshold, whether an index is to be established for the column.", + "F2": "Acquiring index status information of the column within the time threshold to determine current index usage.", + "F3": "Determining, based on index status information, whether a current index is used for the column within the time threshold.", + "F4": "Determining if the number of times the current index has been used within the time threshold is not less than a preset number-of-times threshold.", + "F5": "Establishing an index for the column if the number of times the current index has been used is not less than the number-of-times threshold.", + "F6": "Determining an index type from a plurality of index types according to data information of the column.", + "F7": "Selecting a B-tree index for continuous value type columns and a Hash index when a Join occurs in the column.", + "F8": "Establishing an index for the column according to the determined index type after deciding that an index is to be established.", + "F9": "In response to the determination that an index is not to be established for the column, determining, after the time threshold, whether an index is to be established for the column.", + "F10": "Determining an index is not to be established if the current index is not used within the time threshold or if the usage count is below the threshold.", + "F11": "Including inverted index and Bitmap index in the plurality of index types.", + "F12": "Determining the index type as inverted index if the number of words in the column is greater than a preset number-of-words threshold.", + "F13": "Determining the index type as bitmap index if the number of words is not greater than the preset threshold, no Join occurs, and the column is a discontinuous value type.", + "F14": "Receiving a retrieval expression and dividing it into a plurality of sub-expressions.", + "F15": "Querying whether a retrieval result for a sub-expression exists in a cache.", + "F16": "Generating a retrieval response using the existing retrieval result, or performing retrieval using an index if not existing, and storing the result in the cache.", + "F17": "Initializing the database and constructing a first set of indexes for each column according to a default index type.", + "F18": "Reconstructing a second set of indexes for each column after a preset period of time." + } + } + }, + { + "FAN": "79741091", + "CPC": [ + "G06F-013/00", + "G06F-016/00", + "G06F-016/2458", + "G06F-016/25", + "G06F-016/90335", + "G06F-016/93", + "G06F-016/951", + "G06F-016/953", + "H04L-067/02", + "H04L-067/10", + "H04L-067/12" + ], + "Content": { + "title": "Data accumulation apparatus, data accumulation method, and program", + "pa01": "In one aspect, a data accumulation apparatus includes a script execution environment, an accumulation and retrieval unit accumulating data and performing retrieval on the accumulated data, a notification unit outputting notification in response to that data newly accumulated in the accumulation and retrieval unit matches one of conditions registered in advance, and a storage unit storing scripts and action correspondence information. The script execution environment has property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, according to a script being executed. The data accumulation apparatus further includes a collection unit collecting data through the network and registering the data in the accumulation and retrieval unit, and a script execution unit executing, on receipt of the notification, a script associated with the matched condition according to the action correspondence information using the script execution environment.", + "c-en-0001": "A data accumulation apparatus for temporarily accumulating data received from a first device through a network and transmitting the data to a second device through the network, the data accumulation apparatus comprising: a memory configured to store data providing a script execution environment, scripts, and action correspondence information, the action correspondence information indicating correspondence between the scripts and conditions registered in advance, each of the conditions being associated with a script to be performed when the condition is satisfied, the script execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network according to a script among the scripts that is executed; and a processor coupled to the memory, a combination of the processor and the memory being configured to function as a database including a first function to accumulate data in the database and perform retrieval on the accumulated data, and a second function to output notification in response to new data from the first device being accumulated in the database and matching a condition of the conditions registered in advance, the scripts including a first script configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device and a second script configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event, the processor being configured to function as: a collection unit configured to collect data from the first device through the network and register the data in the database; and a processing unit configured to specify, on receipt of the notification, a script associated with the matched condition according to the action correspondence information, and execute the specified script using the script execution environment.", + "c-en-0002": "The data accumulation apparatus according to claim 1 , wherein the scripts include: an acquisition unit acquiring data from the database in accordance with the instruction to perform data retrieval; and a sending unit sending the data acquired by the acquisition unit to the network in accordance with the instruction to perform communication through the network.", + "c-en-0003": "The data accumulation apparatus according to claim 2 , wherein each of the conditions is a condition matching only part of data in a set of all the data registered by the collection unit, the data acquired by the acquisition unit is one or more pieces of data, and the one or more pieces of data is sent in a state of being stored in a data structure of one of an array, a list, and a nest structure, or sent after identifiers are provided to the one or more pieces of data, when the one or more pieces of data is sent in accordance with the instruction to perform communication through the network.", + "c-en-0004": "The data accumulation apparatus according to claim 1 wherein the script execution environment includes a class library, the class library includes a class to perform data retrieval and a class to perform communication through the network, and the property is achieved by calling the class library with the script being executed.", + "c-en-0005": "The data accumulation apparatus according to claim 1 , wherein the instruction to perform communication through the network uses a HTTP or HTTPS protocol as a transport protocol, and uses a JSON or XML data format as a data exchange code.", + "c-en-0006": "The data accumulation apparatus according to claim 1 , wherein the processing unit sends an accumulation instruction to a document-oriented database in accordance with the instruction to perform communication through the network, the document-oriented database accumulating a document described in a JSON or XML data format.", + "c-en-0007": "The data accumulation apparatus according to claim 1 , wherein the processing unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event, in accordance with the instruction to perform communication through the network.", + "c-en-0008": "The data accumulation apparatus according to claim 3 , wherein the processing unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event, in accordance with the instruction to perform communication through the network, and the notification signal includes identification information to distinguish a data set formed of the one or more pieces of data from another data set.", + "c-en-0009": "The data accumulation apparatus according to claim 1 , wherein both the database and the processing unit are achieved using Light uTupleSpace .", + "c-en-0010": "A data accumulation apparatus for temporarily accumulating data received from a first device through a network and transmitting the data to a second device through the network, the data accumulation apparatus comprising: a memory configured to store data providing a binary execution environment and action correspondence information, the action correspondence information indicating correspondence between conditions registered in advance and processes operating on the binary execution environment, the binary execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, according to the process being executed; and a processor coupled to the memory, a combination of the processor and the memory being configured to function as a database including a first function to accumulate data in the database and perform retrieval on the accumulated data and a second function to output first notification in response to new data from the first device being accumulated in the database and matching a condition of the conditions, the conditions being registered in the database in advance, the processes including a first process configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device, and a second process configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event, the processor being configured to function as: a process execution unit configured to execute the processes using the binary execution environment; a collection unit configured to collect data from the first device through the network and register the data in the database; and a notification unit configured to specify, on receipt of the first notification, a process associated with the matched condition according to the action correspondence information and output second notification to the specified process.", + "c-en-0011": "The data accumulation apparatus according to claim 10 , wherein the processes include: an acquisition unit acquires data from the database, in accordance with the instruction to perform data retrieval; and a sending unit sending the data acquired by the acquisition unit to the network, in accordance with the instruction to perform communication through the network.", + "c-en-0012": "The data accumulation apparatus according to claim 10 , wherein the binary execution environment includes a class library, the class library includes a class to perform data retrieval and a class to perform communication through the network, and the property is achieved by calling the class library with the process being executed.", + "c-en-0013": "A data accumulation method performed by a data accumulation apparatus for temporarily accumulating data received from a first device through a network and transmitting the data to a second device through the network, the data accumulation apparatus comprising: a script execution environment; a database including a first function to accumulate data in the database and perform retrieval on the accumulated data and a second function to output notification in response to new data from the first device being accumulated in the database and matching a condition of conditions registered in advance; and a storage unit storing scripts and action correspondence information, the scripts including a first script configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device and a second script configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event, the action correspondence information indicating correspondence between the scripts and the conditions registered in advance, the script execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, the method comprising: collecting data from the first device through the network and registering the data in the database; determining, when new data is accumulated in the database, whether the new data matches the conditions registered in advance; outputting notification in response to the new data matching a condition of the conditions; and specifying, on receipt of the notification, a script associated with the matched condition according to the action correspondence information, and executing the specified script using the script execution environment.", + "c-en-0014": "A non-transitory computer readable medium comprising a program to cause a computer to implement a data accumulation method performed by a data accumulation apparatus for temporarily accumulating data received from a first device through a network and transmitting the data to a second device through the network, comprising: a script execution environment; a database including a first function to accumulate data in the database and perform retrieval on the accumulated data and a second function to output notification in response to new data from the first device being accumulated in the database and matching a condition of conditions registered in advance; and a storage unit storing scripts and action correspondence information, the scripts including a first script configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device and a second script configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event, the action correspondence information indicating correspondence between the scripts and the conditions registered in advance, the script execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, the method comprising: collecting data from the first device through the network and registering the data in the database; determining, when new data is accumulated in the database, whether the new data matches the conditions registered in advance; outputting notification in response to the new data matching a condition of the conditions; and specifying, on receipt of the notification, a script associated with the matched condition according to the action correspondence information, and executing the specified script using the script execution environment.", + "c-en-0015": "A data accumulation method performed by a data accumulation apparatus for temporarily accumulating data received from a first device through a network and transmitting the data to a second device through the network, the data accumulation apparatus comprising: a binary execution environment; a database including a first function to accumulate data in the database and perform retrieval on the accumulated data and a second function to output first notification in response to new data from the first device being accumulated in the database and matching a condition of conditions registered in advance; and a storage unit storing action correspondence information indicating correspondence between the conditions and processes operating on the binary execution environment, the processes including a first process configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device, and a second process configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event, the binary execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, according to the process being executed, the method comprising: collecting data from the first device through the network and registering the data in the database; when new data is accumulated in the database, determining whether the new data matches the conditions registered in advance, and outputting first notification in response to the new data matching a condition of the conditions registered in advance; and specifying, on receipt of the first notification, a process associated with the matched condition according to the action correspondence information, and outputting second notification to the specified process.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a 371 of U.S. National State of International Application No. PCT/JP2017/040467, filed on Nov. 9, 2017, which claims priority to Japanese Application No. 2016-219472, filed on Nov. 10, 2016. The entire disclosures of the above applications are incorporated herein by reference.", + "p0003": "FIELD", + "p0004": "The present invention relates to a data accumulation apparatus, a data accumulation method, and a program accumulating data such as sensor data.", + "p0005": "BACKGROUND", + "p0006": "In a field called sensor network or IoT , there are many examples of constructing a system in which sensors are connected to a network, pieces of data acquired by the sensors are aggregated and accumulated in a cloud. However, when the system is designed such that respective sensors are connected to the Internet to directly transmit data to the cloud, the system has problems in respect of security, the processing capacity of the sensors, and/or the communication load.", + "p0007": "For this reason, in many cases, the system adopts a structure in which an apparatus aggregating pieces of data is provided between the sensor and the cloud. With the three-stage structure formed of the sensors, the data aggregation apparatus, and the cloud, it is possible to leave securement of security in the Internet communication to the data aggregation apparatus. In addition, because the sensors may communicate with only the data aggregation apparatus, the sensors can be achieved with simple computers, and the problem of the processing capacity is solved.", + "p0008": "In addition, by adding a function of temporarily accumulating data to the data aggregation apparatus, the data aggregation apparatus is enabled to also operate as a buffer temporarily aggregating pieces of data. For example, when the data aggregation apparatus accumulates 60 pieces of data from a temperature sensor, which outputs one piece of data per minute, and transmits the pieces of data for 1 hour together to the cloud, the number of times of communications is reduced, and the problem of the communication load is also solved.", + "p0009": "In addition, in prior art, known techniques include a database technique for a sensor network called uTSLite . uTSLite not only has a function of accumulating data and retrieving later, like ordinary relational databases , but also has a function of registering a condition in advance, automatically determining whether data matches the condition when the data is accumulated later, and notifying the application that the data matches the condition when the data matches the condition. Using such functions reduces the labor to develop an application to set conditional expressions in advance for various abnormal events and perform notification when an abnormal event occurs.", + "p0010": "The existing apparatuses performing data aggregation and accumulation as described above have a problem of scarce multiplicity of use and difficulty in development, as detailed hereinafter. For example, some matters are different between individual application examples of sensor networks. The matters include, for example, which sensors are to be connected, for how many number of times pieces of data are to be aggregated and sent, and to which service on the cloud data is to be sent. For this reason, there is the problem that the apparatuses cannot be constructed as uniform general-purpose apparatuses but have to be developed individually for the respective application examples.", + "p0011": "In particular, this problem causes a marked problem in an application example in which data is specially sent when a specific notable event occurs in the case where pieces of data are aggregated. For example, in the example described above, when an emergency event seemed to be a fire is sensed with the temperature sensor, it is required to send alert signal urgently without waiting for sending performed once per hour. However, because it is entirely different between application examples which sensor is to be connected, and which urgent signal is to be sent according to which condition determination, such processing is not achieved by simply data definition, such as \u201caggregation of pieces of data for N number of times\u201d and \u201caggregation of pieces of data for N seconds\u201d. This reason requires individual development, and causes difficulty in development.", + "p0012": "In addition, even when uTSLite is used, the problem described above still exists. Because uTSLite is a software library, uTSLite alone cannot solve the problem described above with respect to the point that application software using uTSLite have to be newly developed.", + "p0013": "SUMMARY", + "p0014": "The present invention has been made in consideration of the circumstances described above. An object of the present invention is to provide a data accumulation apparatus, a data accumulation method, and a program capable of flexibly defining how pieces of data are aggregated and sent by what condition determination in an apparatus performing aggregation and accumulation of data, such as sensor data.", + "p0015": "In a first aspect of the present invention, a data accumulation apparatus accumulating data received through a network includes a script execution environment, an accumulation and retrieval unit accumulating data and performing retrieval on the accumulated data, a notification unit outputting notification in response to that data newly accumulated in the accumulation and retrieval unit matches one of conditions registered in advance, and a storage unit storing scripts and action correspondence information indicating correspondence between the conditions registered in advance and the scripts, the script execution environment has property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, according to a script being executed, and the data accumulation apparatus further includes a collection unit collecting data through the network and registering the data in the accumulation and retrieval unit, and a script execution unit executing, on receipt of the notification, a script associated with the matched condition according to the action correspondence information using the script execution environment.", + "p0016": "In a second aspect, the scripts include an acquisition unit acquiring data from the accumulation and retrieval unit in accordance with the instruction to perform data retrieval, and a sending unit sending the data acquired by the acquisition unit to the network in accordance with the instruction to perform communication through the network.", + "p0017": "In a third aspect, each of the conditions registered in advance is a condition matching only part of data in a set of all the data registered by the collection unit, the data acquired by the acquisition unit is one or more pieces of data, and the one or more pieces of data is sent in a state of being stored in a data structure of one of an array, a list, a structure, and a nest structure, or sent after identifiers are provided to the one or more pieces of data, when the one or more pieces of data is sent in accordance with the instruction to perform communication through the network.", + "p0018": "In a fourth aspect, the script execution environment includes a class library, the class library includes a class to perform data retrieval and a class to perform communication through the network, and the property is achieved by calling the class library with the script being executed.", + "p0019": "In a fifth aspect, the instruction to perform communication through the network uses a HTTP or HTTPS protocol as a transport protocol, and uses a JSON or XML data format as a data exchange code.", + "p0020": "In a sixth aspect, the script execution unit sends an accumulation instruction to a document-oriented database in accordance with the instruction to perform communication through the network, the document-oriented database accumulating a document described in the JSON or XML data format.", + "p0021": "In a seventh aspect, the script execution unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event, in accordance with the instruction to perform communication through the network.", + "p0022": "In an eighth aspect, the script execution unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event, in accordance with the instruction to perform communication through the network, and the notification signal includes identification information to distinguish a data set formed of the one or more pieces of data from another data set.", + "p0023": "In a ninth aspect, both the accumulation and retrieval unit and the notification unit are achieved using uTSLite.", + "p0024": "In a tenth aspect, a data accumulation apparatus accumulating data received through a network includes a binary execution environment, an accumulation and retrieval unit accumulating data and performing retrieving on the accumulated data, a first notification unit outputting first notification in response to that data newly accumulated in the accumulation and retrieval unit matches one of conditions registered in advance, and a storage unit storing action correspondence information indicating correspondence between the conditions registered in advance and processes operating on the binary execution environment, the binary execution environment has property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network, according to the process being executed, and the data accumulation apparatus further includes a collection unit collecting data through the network and registering the data in the accumulation and retrieval unit, and a second notification unit outputting, on receipt of the first notification, second notification to a process associated with the matched condition according to the action correspondence information.", + "p0025": "In an eleventh aspect, the processes include an acquisition unit acquires data from the accumulation and retrieval unit, in accordance with the instruction to perform data retrieval, and a sending unit sending the data acquired by the acquisition unit to the network, in accordance with the instruction to perform communication through the network.", + "p0026": "In a twelfth aspect, each of the conditions registered in advance is a condition matching only part of data in a set of all the data registered by the collection unit, the data acquired by the acquisition unit is one or more pieces of data, and the one or more pieces of data is sent in a state of being stored in a data structure of one of an array, a list, a structure, and a nest structure, or sent after any identifiers are provided to the one or more pieces of data, when the data is sent in accordance with the instruction to perform communication through the network.", + "p0027": "In a thirteenth aspect, the binary execution environment includes a class library, the class library includes a class to perform data retrieval and a class to perform communication through the network, and the property is achieved by calling the class library with the process being executed.", + "p0028": "In a fourteenth aspect, the instruction to perform communication through the network uses a HTTP or HTTPS protocol or a MQTT protocol as a transport protocol, and uses a JSON or XML data format as a data exchange code.", + "p0029": "In a fifteenth aspect, the data accumulation apparatus further includes a process execution unit executing the processes using the binary execution environment, and the process execution unit sends an accumulation instruction to a database in accordance with the instruction to perform communication through the network, the database accumulating data or a file described in the JSON or XML data format.", + "p0030": "In a sixteenth aspect, the data accumulation apparatus further includes a process execution unit executing the processes using the binary execution environment, the process execution unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event, in accordance with the instruction to perform communication through the network, and the notification signal includes identification information to distinguish a data set formed of the one or more pieces of data from another data set.", + "p0031": "The present invention enables flexible definition as to how pieces of data are aggregated and sent by what condition determination in an apparatus performing aggregation and accumulation of data such as sensor data.", + "p0032": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0033": "FIG. 1 is a block diagram illustrating configuration of a data management system according to a first embodiment of the present invention;", + "p0034": "FIG. 2 is a flowchart illustrating sensor data registration processing of a data accumulation apparatus illustrated in FIG. 1 ;", + "p0035": "FIG. 3 is a diagram illustrating an example of a script;", + "p0036": "FIG. 4 is a diagram illustrating an example of content of communication from the data accumulation apparatus to a cloud DB;", + "p0037": "FIG. 5 is a diagram illustrating another example of the script;", + "p0038": "FIG. 6 is a block diagram illustrating configuration of a data management system according to a second embodiment of the present invention;", + "p0039": "FIG. 7 is a flowchart illustrating sensor data registration processing of a data accumulation apparatus illustrated in FIG. 6 ; and", + "p0040": "FIG. 8 is a block diagram illustrating a hardware configuration example of a data accumulation apparatus according to an embodiment of the present invention.", + "p0041": "DETAILED DESCRIPTION", + "p0042": "Embodiments of the present invention will be described hereinafter with reference to drawings.", + "p0043": "First Embodiment", + "p0044": "FIG. 1 illustrates configuration of a data management system 100 according to a first embodiment of the present invention. In the data management system 100 , a data accumulation apparatus 101 , a sensor 103 , a cloud DB 104 , and a tablet 105 are connected to a network 102 . Connection thereof to the network 102 may be wired or wireless connection. The sensor 103 performs measurement, for example, at regular time intervals, and sends a measurement result to the data accumulation apparatus 101 through the network 102 . The data accumulation apparatus 101 accumulates the data collected from the sensor 103 through the network 102 . In the present embodiment, the system includes one sensor 103 to simplify the explanation, but a plurality of sensors 103 may be connected to the network 102 .", + "p0045": "The data accumulation apparatus 101 retains at least one condition registered in advance. When new data is accumulated, the data accumulation apparatus 101 determines whether the data matches each of the conditions, and performs processing corresponding to the matched condition. For example, the data accumulation apparatus 101 sends a notification signal to the cloud DB 104 or the tablet 105 through the network 102 . The notification signal may include, for example, part of data accumulated in the data accumulation apparatus 101 . The cloud DB 104 is implemented on, for example, a server device not illustrated. The tablet 105 is an example of the information processing apparatus. Examples of the information processing apparatus include a personal computer , a server device, a smartphone, a mobile phone, and the like. A plurality of tablets 105 may be connected to the network 102 .", + "p0046": "The data accumulation apparatus 101 is formed of an information processing apparatus, such as a server device or a PC, and includes a processor, such as a CPU , a memory, and a communication interface 119 to perform communication through the network 102 , as hardware. The data accumulation apparatus 101 may further include an interface for input and output, such as a display, a keyboard, or a mouse. The conditions described above may be set through the interface for input and output, or set from the outside through the network 102 , for example. The conditions are stored in the memory.", + "p0047": "The data accumulation apparatus 101 includes a lightweight stream DB 111 , a sensor data collection unit 112 , a script execution environment 115 , a class library 116 , an initialization unit 117 , and a processing unit 118 , and retains action correspondence information 113 and scripts 114 . Various types of processing performed by the lightweight stream DB 111 , the sensor data collection unit 112 , the initialization unit 117 , and the processing unit 118 can be achieved by reading and executing various programs from the memory with the processor. The data accumulated in the lightweight stream DB 111 , the class library 116 , the action correspondence information 113 , and the scripts 114 are stored in the memory.", + "p0048": "In the present embodiment, uTSLite is used as the lightweight stream DB 111 . uTSLite has an accumulation and retrieval function to accumulate data and perform retrieval on the accumulated data, and a notification function to output notification when newly accumulated data matches any of the conditions registered in advance. Specifically, the accumulation and retrieval function is a function to: perform retrieval processing of a data value accumulation type including accumulating data in a data depository, and inputting a conditional expression later, thereby retrieving data matching a conditional expression from the data depository; and notify a retrieval result. The notification function is a function to: perform retrieval processing of a conditional expression waiting type including accumulating conditional expressions in a conditional expression depository, and writing data to a data depository later, thereby retrieving the conditional expression matching the data from the conditional expression depository; and notify a retrieval result.", + "p0049": "The scripts 114 are described in a JavaScript language, and a JavaScript execution environment is used as the script execution environment 115 . The JavaScript execution environment can be easily achieved by using open source software, such as Node.js. In addition, the class library 116 is registered in advance in the script execution environment 115 using a standard function of the JavaScript execution environment. In this manner, the class library 116 can be called by the description in the script 114 . The class library 116 provides a class to retrieve data from the lightweight stream 111 , and a class to perform communication through the network 102 by a HTTP or HTTPS protocol. In other words, the script execution environment 115 has property of being capable of describing an instruction to retrieve data from the lightweight stream DB 111 and an instruction to perform communication through the network 102 by the HTTP or HTTPS protocol, according to the script 114 being executed.", + "p0050": "The sensor data collection unit 112 has a function to wait for connection by HTTP, that is, serves as an HTTP server. The sensor data collection unit 112 receives sensor data from the sensor 103 through the network 102 , and registers the sensor data in the lightweight stream DB 111 .", + "p0051": "Software of a type generally called \u201cdocument-oriented DB\u201d is used as the cloud DB 104 . For example, Elastic search being open source software may be used. A document-oriented DB is capable of accumulating documents described in JSON or XML data format.", + "p0052": "An ordinary Android terminal in which DeviceConnect being open source software is installed is used as the tablet 105 . DeviceConnect is software providing a function of receiving a notification signal from the outside and drawing alert display on the screen, or displaying web content of designated URL on the browser. Another software having a function equal to DeviceConnect may be used.", + "p0053": "The initialization unit 117 registers conditional expressions in the lightweight stream DB 111 and retains the scripts 114 , and performs processing to retain correspondence between the conditional expressions and the scripts 114 as the action correspondence information 113 , when the data accumulation apparatus 101 is started up. For example, the correspondence to associate the conditional expression \u201ctemperature>80\u00b0 C.\u201d with the script \u201caction1.js\u201d and associate the conditional expression \u201cintruder=true\u201d with the script \u201caction2.js\u201d is described in the definition file to be read in startup. The initialization unit 117 registers the conditional expression \u201ctemperature>80\u00b0 C.\u201d in the lightweight stream DB 111 , reads the contents of the file \u201caction1.js\u201d as the script 114 , and retains it on the memory. In the same manner, the initialization unit 117 registers the conditional expression \u201cintruder=true\u201d in the lightweight stream DB 111 , reads the contents of the file \u201caction2.js\u201d as the script 114 , and retains it on the memory. The initialization unit 117 also retains correspondence between the conditional expressions and the scripts 114 on the memory as follows, as information corresponding to the action correspondence information 113 .", + "p0054": "As the action correspondence information 113 , for example, in the example described above, a data structure is retained on the memory such that the correspondence \u201caction1.js\u201d is obtained when it is notified that the data matches the conditional expression temperature>80\u00b0 C. For example, because uTSLite has the function of calling the designated function with the designated identifier in notification, it is easily determined to which conditional expression the event notification corresponds, by using a language function, such as the class object, the associative array, or the lambda expression in the C++ language. For example, in the example described above, the correspondence of \u201caction1.js\u201d is easily obtained when the data matches the condition temperature>80\u00b0 C. In this case, the information secured on the memory with the function of the C++ language, such as the class object, the associative array, or the lambda expression, and the information for event notification retained in the uTSLite correspond to the action correspondence information 113 . Like this example, it is possible to adopt implementation as a matter of course, in which no explicit data structure or storage exists, such as an array corresponding to the action correspondence information 113 , and corresponding information is retained in a fragmentary state on the memory.", + "p0055": "The processing unit 118 receives the notification from the lightweight stream DB 111 , specifies the script 114 associated with the conditional expression serving as a trigger to issue the notification with reference to the action correspondence information 113 , and executes the script 114 using the script execution environment 115 .", + "p0056": "FIG. 2 illustrates an example of flow of sensor data registration processing in the data accumulation apparatus 101 .", + "p0057": "When the sensor 103 transmits sensor data of a measurement result to the data accumulation apparatus 101 through the network 102 using the HTTP protocol, the sensor data collection unit 112 accepts HTTP connection. Specifically, the sensor data collection 112 receives sensor data from the sensor 103 , and accumulates the received data in the lightweight stream DB 111 . The accumulation processing at Step S 201 is easily achieved by calling the data registration function provided by the lightweight stream DB 111 , with the received data serving as an argument. In this operation, when the data matches the conditional expression registered in advance, a notification is output from the lightweight stream DB 111 , and the processing unit 118 receives the notification . With the function of the lightweight stream DB 111 , the notification includes data accumulated at Step S 201 , that is, data serving as the notification trigger. Thereafter, the processing unit 118 specifies the corresponding script 114 based on the action correspondence information 113 . Thereafter, the processing unit 118 executes the script 114 specified at Step S 204 using the script execution environment 115 . In this operation, the processing unit 118 executes the script after setting the data serving as the notification trigger as the value of the variable \u201cevent\u201d readable in the script. Lastly, the processing unit 118 ends the processing on the notification .", + "p0058": "When the data matches N conditional expressions, because the data accumulation apparatus 101 a notification from the lightweight stream DB 111 N times, the processing from Step S 203 to Step S 206 described above is repeatedly executed N times. N is an integer of 0 or more.", + "p0059": "The data accumulation apparatus 101 according to the present embodiment performs the basic operation described above, thereby describing various actions for various conditions designated at initialization by the program of JavaScript. Thus, it is possible to achieve various functions by only replacing the setting files in a single developed apparatus. In particular, because retrieval of the lightweight stream DB 111 and HTTP or HTTPS communications to various communication parties is easily described from JavaScript using the function of the class library 116 , it is possible to flexibly and easily achieve the functions required in an apparatus performing aggregation and accumulation of information, such as sensor data.", + "p0060": "The following explanation further illustrates some examples of script description, to specifically describe the advantage of the data accumulation apparatus 101 according to the present embodiment.", + "p0061": "FIG. 3 illustrates an example of the script. A script 300 illustrated in FIG. 3 achieves an operation of retrieving a set of data in which the value of the key \u201csubject\u201d is the character string \u201csensor\u201d, the value of the key \u201ctype\u201d is the character string \u201cthermometer\u201d, and the time stamp having the precision of milliseconds and stored as the value of the key \u201cdate\u201d falls within the range of time from 60 seconds before the present to the present in the data accumulated in the lightweight stream DB 111 , and registering the set of data as one data entry in the cloud DB 104 .", + "p0062": "First, with the row group 301 , the value of the current time and the value of the time of 60 seconds before the current time are obtained. Thereafter, the class library \u201cajax.collect\u201d is called with the row 302 , and a result is obtained by searching the lightweight stream DB 111 . The library \u201cajax.collect\u201d is one of class functions provided from the class library 116 . Implementation of this class is easily achieved by substituting the provided argument for the variable type used for input and output of the lightweight stream DB 111 , and calling the function of the lightweight stream DB 111 .", + "p0063": "When the retrieval result is not empty, the class library \u201cajax.request\u201d is called with the row 303 , and HTTP POST communication is sent to register the result in the cloud DB 104 . The library \u201cajax.request\u201d is one of class functions provided from the class library 116 . Implementation of this class is easily achieved by setting the contents of the provided arguments \u201ctype\u201d and \u201cheaders\u201d as the header of the HTTP protocol for the host of the URL indicated with the provide argument \u201curl\u201d, and using the contents of the provided argument \u201cdata\u201d as the content. The processing to perform HTTP communication may be performed using OSS which is widely provided. In the row 303 , the retrieval result is stored as the argument \u201cdata\u201d. The retrieval result is formed of a string of pieces of data. With the standard function included in JavaScript and the JSON format and enabling description of pieces of data in a nest manner, a string of a plurality of pieces of data can be dealt with as a nest value. The content is reflected on the communication content as they are. In addition, in JavaScript, the language itself has a function of automatically converting the data into the JSON format in input and output of data with the outside.", + "p0064": "Although it is not used in the example of the script 300 , because the value of the variable \u201cevent\u201d stores data serving as the notification trigger as described above, the script can be described to use the data. For example, it is easy to describe a script that uses the time stamp of data serving as the notification trigger as the value of the identifier, and provides the value to all the other pieces of data, to distinguish them from other pieces of data later.", + "p0065": "FIG. 4 illustrates an example of communication content to the cloud DB 104 . A message 400 illustrated in FIG. 4 indicates content stored in the body in the HTTP protocol. As described above, the content stores a string of data serving as a retrieval result with the row 303 of the script 300 illustrated in FIG. 3 . Because it is data accumulated in the DB that is obtained by search of the DB, the data received from the sensor 103 at Step S 200 before and accumulated at Step S 201 is reflected as it is on the message 400 . For example, the row 401 indicates data in which the key \u201cdate\u201d has a value of 1473240068, the key \u201csubject\u201d has a value \u201csensor\u201d, the key \u201ctemperature\u201d has a value of 27.53125, and the key \u201ctype has a value \u201cthermometer\u201d.", + "p0066": "The example of the message 400 is expressed with array data in which five data entries are arranged in a nest manner in the JSON format. In the case where Elasticsearch is used as the cloud DB 104 , when such a message is received, the cloud DB 104 has the operation specification in which five data entries are generated. The number of data entries are not limited to five, but may be 1, 2 to 4, or 6 or more.", + "p0067": "The data is not limited to a nest structure, but may be stored and sent in another data structure . The data may be sent after each of pieces of data is provided with any identifier. This enables the receiving side of the network 102 to recognize that the pieces of data are sent together as a set.", + "p0068": "FIG. 5 illustrates another example of the script. A script 500 illustrated in FIG. 5 achieves an action to perform notification on the tablet 105 and promote browser display, by transmitting a request in which time information indicating the time range from the time of 60 seconds before the current time to the current time is embedded in the instruction sentence to the tablet 105 .", + "p0069": "First, the value of the current time and the value of the time of 60 seconds before the current time are obtained with a row group 501 . Thereafter, the class library \u201cajax.request\u201d is called with the row 502 , and HTTP POST communication is sent to the tablet 105 . In this operation, the IP address of the tablet 105 is designated with the argument \u201curl\u201d. In addition, as the content of the argument \u201cdata\u201d serving as the content of HTTP, the character string in which the time information calculated with the row group 501 is embedded is generated and sent.", + "p0070": "DeviceConnect installed in the tablet 105 used in the present embodiment has operation specification of first displaying notification on the terminal when it receives HTTP content of such form, and displaying web content of the URL designated with the content when the notification icon is tapped.", + "p0071": "By embedding the time information together with the URL, the displayed web content is enabled to perform processing using the time information. For example, it is possible to easily prepare web content to retrieve data indicated in the range of the time information from the cloud DB 104 , and displays the data as a line graph.", + "p0072": "As described above, in the present embodiment, it is possible to register pairs of correspondence of the conditional expressions to the scripts in initialization. The conditional expressions of the respective pairs may be different, or the same conditional expression may be used in some pairs of correspondence. For this reason, the correspondence in which the script 300 is associated with the conditional expression, and the correspondence in which the script 500 is associated with the same conditional expression are registered in initialization. In this manner, data of the sensor is registered with an event occurring at the certain time, the conditional expressions become true, and the script 300 and the script 500 are successively executed without time interval. Because the range of the time stamp of data serving as the retrieval target in the row 302 of the script 300 agrees with the time range included in the message in the row 502 of the script 500 , the web content displayed on the tablet that has received a notification signal with the script 500 is enabled to display the data serving as the retrieval target as a graph. As described above, according to the present embodiment, the time stamp information embedded in the sensor data as a matter of course is also used as the identifier, to enable recognition of correspondence when the data registered in the cloud DB 104 is displayed as the web content later.", + "p0073": "The scope of the present invention is not limited thereto. The present invention may be achieved with a structure in which a plurality of pieces of data is registered as, for example, a long data entry, in the cloud DB 104 , or pieces of data are provided with respective keys in new JSON data format and specific identifiers are provided as the values of the keys. In particular, the latter is easily achieved by adding a loop sentence to patrol each element of the variable \u201cthermometer\u201d and adds a key before the row 303 of the script 300 . The value included in the variable \u201cevent\u201d may be used as the key.", + "p0074": "The following is an explanation of specific effects obtained according to the present embodiment described above.", + "p0075": "An advantage of reduction in communication quantity by aggregation of registration of sensor data is obtained by describing a condition with low establishment frequency as the conditional expression, performing retrieval in a range to acquire a number of entries from the lightweight stream DB 111 with the script associated with the conditional expression, and sending a result of retrieval to the cloud DB 104 . For example, when 60 pieces of data from a temperature sensor transmitted once per minute are accumulated to transmit pieces of information for one hour together to the cloud DB 104 , the number of times of communication is reduced, and the communication load is reduced. As another example, when the temperature unusually increases, the number of times of communications can be reduced by transmitting several pieces of data directly before the unusual increase in a batch to the cloud DB 104 . This enables reduction in communication load. In addition, when the temperature unusually increases, data is transmitted to the cloud without waiting for execution of processing to batch pieces of data for one hour. This enables acquisition of an advantage that abnormality can be promptly sensed while the communication load is reduced.", + "p0076": "These conditional expressions and setting as to what cloud DB 104 the retrieval conditions are to be set can be easily customized by changing the file to be read in initialization. This enables flexible change of the definition, and reduction in the number of steps to develop the system of the sensor network using the apparatus of the present invention.", + "p0077": "In the present embodiment, the single class library \u201cajax.request\u201d achieves both data registration to the cloud DB 104 in the row 303 of the script 300 and notification for the tablet 105 in the row 502 of the script 500 . This is because both the cloud DB 104 and the tablet 105 used in the present embodiment provide an API to achieve functions, such as data registration and notification display, by receiving communication by HTTP. As described above, because the HTTP serving as a web technique or the API obtained by combining HTTP with JSON has become widespread in the technical field called cloud or IoT, the apparatus like the data accumulation apparatus 101 according to the present embodiment produces an advantage of being capable of being combined with a wide variety of apparatuses with a single mechanism by adopting HTTP and JSON as a standard input and output. In the present embodiment, because the communication function from the script 114 is provided from the class library 116 , the present embodiment has an advantage that the apparatus can be expanded to a communication party including various protocols as the API by expansion of the class library 116 . In addition, it is possible to connect with many parties without expansion of the class library 116 by providing communications by HTTP and JSON in advance as a standard function as the class library 116 .", + "p0078": "In the present embodiment, uTSLite is used as the lightweight stream 111 . This structure produces an advantage that a single functional module achieves the function of accumulating data and retrieving data later, and the function of performing notification when the data accumulated later matches the condition registered in advance. The scope of the present invention is not limited thereto. Another DB system may be used, or the respective functions may be achieved separately. For example, the function of accumulating data and retrieving the data later may be achieved using a relational database. The function of performing notification when the data accumulated later matches the condition registered in advance may be achieved using a technique, such as an array, a list, an associative array, a tree structure, a Pub-Sub system, a stored procedure, or a stream DB.", + "p0079": "In the present embodiment, JavaScript language is used for the script 114 . The JavaScript language is widely used when communication to the cloud is described in combination with HTTP and JSON, and has an advantage that various existing libraries can be used. The scope of the present invention is not limited thereto, but another programming language may be used.", + "p0080": "In the present embodiment, JSON is used as the data form of the data exchange code. The scope of the present invention is not limited thereto, but another data form may be used. In particular, in the document-oriented DB, XML is also widely used as the data form. Using XML as the data form produces an advantage of increase in type of the cloud DBs that can be used in combination.", + "p0081": "The present embodiment illustrates that the data accumulation apparatus 101 is achieved with one information processing apparatus. In another embodiment, the data accumulation apparatus 101 may be achieved with a combination of a plurality of information processing apparatuses.", + "p0082": "Second Embodiment", + "p0083": "FIG. 6 illustrates configuration of a data management system 600 according to a second embodiment of the present invention. In FIG. 6 , elements similar to the elements illustrated in FIG. 1 are denoted by the same reference numerals, and an explanation of the elements will be properly omitted. In the data management system 600 , a data accumulation apparatus 601 , the sensor 103 , the cloud DB 104 , and an apparatus 600 are connected to the network 102 . The apparatus 605 is an example of the information processing apparatus. The apparatus 605 may have the same structure as that of the tablet 105 explained in the first embodiment. The apparatus 605 includes a sensor 606 . The sensor 606 performs measurement to generate sensor data, and transmits the sensor data to the data accumulation apparatus 601 through the network 102 .", + "p0084": "The data accumulation apparatus 601 is an information processing apparatus, such as a server device or a PC, and includes a processor, such as a CPU, a memory, and the communication interface 119 to preform communication through the network 102 , as hardware. The data accumulation apparatus 601 may further include an interface for input and output, for example, a display, a keyboard, and a mouse.", + "p0085": "The data accumulation apparatus 601 includes the lightweight stream DB 111 , the sensor data collection unit 112 , action correspondence information 613 , a binary execution environment 615 , the initialization unit 117 , and a processing unit 618 . The processing unit 618 includes a notification unit 619 and a process execution unit 620 . Various types of processing performed by the lightweight stream DB 111 , the sensor data collection unit 112 , and the initialization unit 117 may be achieved by reading and executing various programs from the memory with the processor.", + "p0086": "The binary execution environment 615 includes a class library similar to the class library 116 explained in the first embodiment. For example, the class library provides a class to retrieve data from the lightweight stream DB 111 , and a class to perform communication with the HTTP or HTTPS protocol or a MQTT protocol through the network 102 . In other words, the binary execution environment 615 has property of being capable of describing an instruction to perform data retrieval from the lightweight stream DB 111 and an instruction to perform communication with the HTTP or HTTPS protocol or MQTT through the network 102 , by a process 614 operating on the binary execution environment 615 .", + "p0087": "The process execution unit 620 executes the process 614 using the binary execution environment 615 . In this manner, the process 614 operates on the binary execution environment 615 . The process 614 is generated by input of a source file to the data accumulation apparatus 601 through the communication interface 119 or the interface for input and output and compiling of the source file on the data accumulation apparatus 601 . The action correspondence information 613 indicates correspondence between the conditions registered in advance and the processes 614 operating on the binary execution environment 615 .", + "p0088": "On receipt of notification from the lightweight stream DB 111 , the notification unit 619 specifies the process 614 associated with the condition serving as a trigger of issuance of the notification with reference to the action correspondence information 613 , and outputs notification to the specified process 614 . The process 614 that has received the notification from the notification unit 619 performs predetermined processing. The processes 614 may perform processing similar to those of the scripts 114 explained in the first embodiment. For example, the process 614 performs processing to search the lightweight stream DB 111 , and a processing to transmit a notification signal notifying occurrence of an event and including data acquired by retrieval to the external device through the network 102 .", + "p0089": "Instead of the implementation described above, the process execution unit 620 may specify the process 614 associated with the condition serving as a trigger of occurrence of the notification with reference to the action correspondence information 613 , on receipt of notification from the lightweight stream DB 111 , and execute the specified process 614 using the binary execution environment 615 . In this case, the process 614 performs predetermined processing when it is started.", + "p0090": "When the data accumulation apparatus 601 boots up, the initialization unit 117 registers conditions in the lightweight stream DB 111 , causes the process execution unit 620 to execute the processes 614 , and causes the memory to retain the action correspondence information 613 indicating correspondence between the conditions and the processes 614 . The action correspondence information 613 may be edited based on an instruction from the process 614 while the data accumulation apparatus 601 is active.", + "p0091": "FIG. 7 illustrates an example of flow of sensor data registration processing in the data accumulation apparatus 601 .", + "p0092": "When the sensor 103 transmits sensor data to the data accumulation apparatus 601 through the network 102 , the sensor data collection unit 112 receives sensor data from the sensor 103 , and accumulates the received sensor data in the lightweight stream DB 111 .", + "p0093": "When the sensor data newly accumulated in the lightweight stream DB 111 matches any condition, the lightweight stream DB 111 outputs notification indicating the condition which the sensor data newly accumulated in the lightweight stream DB 111 matches to the notification unit of the processing unit 618 . The notification unit 619 of the processing unit 618 specifies the process 614 associated with the condition with reference to the action correspondence information 613 . Thereafter, the notification unit 619 outputs notification to the specified process 614 . The process execution unit 620 executing the specified process 614 receives the notification from the notification unit 619 , and searches the lightweight stream DB 111 and/or sends a notification signal through the network 102 .", + "p0094": "The data accumulation apparatus 601 according to the present embodiment is enabled to achieve various functions only by switching the setting file in a single developed apparatus, in the same manner as the first embodiment.", + "p0095": "FIG. 8 illustrates an example of hardware configuration of a data accumulation apparatus 700 according to one embodiment. The data accumulation apparatus 700 may be the data accumulation apparatus 101 or the data accumulation apparatus 601 . As illustrated in FIG. 8 , the data accumulation apparatus 800 includes a CPU 801 , a main memory 802 , a program memory 803 , an auxiliary storage device 804 , a communication interface 805 , and an external interface 806 , which are connected through a bus 807 .", + "p0096": "The CPU 801 reads a program stored in the program memory 803 , loads the program onto the main memory 802 , and executes the program to achieve the functions described above with respect to the data accumulation apparatus 101 or the data accumulation apparatus 601 . The main memory 802 is, for example, a SRAM or a DRAM . The program memory 803 may be a ROM , or may be implemented as part of the auxiliary storage device 804 . The auxiliary storage device 804 is, for example, a HDD or a SSD, and stores various types of data.", + "p0097": "The communication interface 805 includes a wired communication module, a wireless communication module, or a combination thereof, and communicates with the external device . The external interface 806 is an interface to connect with an input device, such as a keyboard, and an output device, such as a display device.", + "p0098": "The CPU 801 is an example of the processor. The processor is not limited to a general-purpose processing circuit like the CPU 801 . The processor may be a dedicated processing circuit, such as an ASIC or a FPGA . When a dedicated processing circuit is used as the processor, the program may exist in the dedicated processing circuit. The processor may include one or more general-purpose processing circuits and/or one or more dedicated processing circuits.", + "p0099": "The program for achieving the processing described above may be stored and provided in a computer-readable recording medium. The program is stored as a file of an installable or executable format in a recording medium. Examples of the recording medium that can be used include a magnetic disk, an optical disk , a magneto-optical disk , and a semiconductor memory. Any recording medium may be used as long as it can store a program and is computer-readable. The program for achieving the processing described above may be stored on a computer connected to a network, such as the Internet, and downloaded to the data accumulation apparatus 800 through the network.", + "p0100": "The sensor data referred to in the present invention includes real-world information measured by a sensor device, and other various information can serve as the target. As specific examples, it is possible to deal with values measured by a sensor device, such as the temperature, the humidity, the current or voltage value, the flow rate of fluid, the density, the brightness, the noise, the position, and the acceleration of a substance. The sensor data are not limited thereto, but may be information acquired by an object other than a sensor, such as information acquired through the web and/or the Internet. In addition, the sensor data may be information including metadata, such as the property and the state of the sensor and the measurement date and time, in addition to those values.", + "p0101": "The present invention is not limited to each of the embodiments described above, but may be embodied with the elements modified without departing from the spirits of the present invention. Various inventions can be formed with proper combinations of the elements disclosed in the embodiments described above. For example, some elements may be deleted from the elements disclosed in each of the embodiments. In addition, the elements of the different embodiments may properly be used in combination.", + "features": { + "F1": "A data accumulation apparatus temporarily accumulates data received from a first device through a network and transmits the data to a second device through the network.", + "F2": "The data accumulation apparatus comprises a memory configured to store data providing a script execution environment, scripts, and action correspondence information.", + "F3": "The action correspondence information indicates correspondence between the scripts and conditions registered in advance, where each condition is associated with a script to be performed when the condition is satisfied.", + "F4": "The script execution environment is capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network according to a script that is executed.", + "F5": "A processor is coupled to the memory, where the combination of the processor and the memory is configured to function as a database.", + "F6": "The database includes a first function to accumulate data and perform retrieval on the accumulated data.", + "F7": "The database includes a second function to output notification in response to new data from the first device being accumulated and matching a condition registered in advance.", + "F8": "The scripts include a first script configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data to the second device.", + "F9": "The scripts include a second script configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event.", + "F10": "The processor functions as a collection unit configured to collect data from the first device through the network and register the data in the database.", + "F11": "The processor functions as a processing unit configured to specify, on receipt of the notification, a script associated with the matched condition according to the action correspondence information, and execute the specified script using the script execution environment.", + "F12": "The scripts include an acquisition unit acquiring data from the database in accordance with the instruction to perform data retrieval.", + "F13": "The scripts include a sending unit sending the data acquired by the acquisition unit to the network in accordance with the instruction to perform communication through the network.", + "F14": "Each condition is a condition matching only part of data in a set of all the data registered by the collection unit.", + "F15": "The data acquired by the acquisition unit is sent in a state of being stored in a data structure of one of an array, a list, and a nest structure, or sent after identifiers are provided.", + "F16": "The script execution environment includes a class library with a class to perform data retrieval and a class to perform communication through the network.", + "F17": "The instruction to perform communication through the network uses a HTTP or HTTPS protocol and a JSON or XML data format.", + "F18": "The processing unit sends an accumulation instruction to a document-oriented database accumulating a document described in a JSON or XML data format.", + "F19": "The processing unit sends a notification signal to notify an information processing apparatus connected to the network of occurrence of an event.", + "F20": "The notification signal includes identification information to distinguish a data set formed of the one or more pieces of data from another data set.", + "F21": "Both the database and the processing unit can be achieved using Light uTupleSpace.", + "F22": "A binary execution environment having property of being capable of describing an instruction to perform data retrieval and an instruction to perform communication through the network.", + "F23": "A processor coupled to the memory functions as a database including a first function to accumulate data and perform retrieval and a second function to output first notification.", + "F24": "Processes include a first process configured to retrieve data within a predetermined time range from the database and transmit a notification signal including the retrieved data.", + "F25": "Processes include a second process configured to transmit a notification signal to notify a third device connected to the network of occurrence of an event.", + "F26": "The method comprises collecting data from the first device through the network and registering the data in the database.", + "F27": "The method comprises determining, when new data is accumulated in the database, whether the new data matches the conditions registered in advance.", + "F28": "The method comprises outputting notification in response to the new data matching a condition of the conditions.", + "F29": "The method comprises specifying, on receipt of the notification, a script associated with the matched condition according to the action correspondence information, and executing the specified script using the script execution environment.", + "F30": "A non-transitory computer readable medium comprises a program to implement a data accumulation method using the described features." + } + } + }, + { + "FAN": "86860100", + "CPC": [ + "G06F-016/00", + "G06F-016/215", + "G06F-016/2237", + "G06F-016/2365", + "G06F-016/2468", + "G06F-017/30/303", + "G06F-017/30/324", + "G06F-017/30/371", + "G06F-017/30/542", + "G06F-021/60/2", + "G06F-021/62/27", + "G06Q-030/01", + "H04L-063/0428" + ], + "Content": { + "title": "Detect duplicates with exact and fuzzy matching on encrypted match indexes", + "pa01": "Disclosed herein are system, method, and computer program product embodiments for detecting duplicates with exact and fuzzy matching on encrypted match indexes using an encryption key in a cloud computing platform. An embodiment operates by determining a match rule index value upon reception of a new record. The embodiment encrypts the match index rule value using the customer's encryption key and a deterministic encryption method and stores the encrypted match rule index value. Duplicate detection may be later performed by using the same deterministic encryption method to determine a cypher text for a candidate entry and comparing the ciphertext to the stored encrypted match indexes.", + "c-en-0001": "A method, comprising: receiving, by one more processors, a match rule from a user of a customer relationship management tool, wherein the match rule comprises a unique identifier, a table in the customer relationship management tool, and one or more fields of applicability; and creating, by the one or more processors, a match index column in the table comprising encrypted match indexes using the unique identifier as an initialization vector in an encryption scheme, wherein the match index column is used when performing duplicate detection to determine if a new record duplicates an encrypted match index in the encrypted match indexes.", + "c-en-0002": "The method of claim 1 , further comprising: in response to receiving the new record for the table, calculating, by the one or more processors, a match index for the new record based on the one or more fields of applicability; and when encryption is enabled on any of the one or more fields of applicability, encrypting the match index for the new record using the encryption scheme to derive a new encrypted match index.", + "c-en-0003": "The method of claim 2 , further comprising: comparing, by the one or more processors, the new encrypted match index to the encrypted match indexes in the match index column; and when no duplicate is found, adding the new record to the table.", + "c-en-0004": "The method of claim 2 , further comprising: comparing, by the one or more processors, the new encrypted match index to the encrypted match indexes in the match index column; and when a duplicate is found, displaying an error message.", + "c-en-0005": "The method of claim 1 , wherein the match rule is specified by the user using Boolean logic.", + "c-en-0006": "The method of claim 1 , where the encryption scheme is a deterministic encryption scheme.", + "c-en-0007": "The method of claim 1 , wherein the encryption scheme is a probabilistic encryption scheme.", + "c-en-0008": "A system, comprising: a memory; and at least one processor coupled to the memory and configured to: receive a match rule from a user of a customer relationship management tool, wherein the match rule comprises a unique identifier, a table in the customer relationship management tool, and one or more fields of applicability; and create a match index column in the table comprising encrypted match indexes using the unique identifier as an initialization vector in an encryption scheme, wherein the match index column is used when performing duplicate detection to determine if a new record duplicates an encrypted match index in the encrypted match indexes.", + "c-en-0009": "The system of claim 8 , the at least one processor further configured to: in response to receiving the new record for the table, calculate a match index for the new record based on the one or more fields of applicability; and when encryption is enabled on any of the one or more fields of applicability, encrypt the match index for the new record using the encryption scheme to derive a new encrypted match index.", + "c-en-0010": "The method of claim 9 , the at least one processor further configured to: compare the new encrypted match index to the encrypted match indexes in the match index column; and when no duplicate is found, add the new record to the table.", + "c-en-0011": "The method of claim 9 , the at least one processor further configured to: compare the new encrypted match index to the encrypted match indexes in the match index column; and when a duplicate is found, display an error message.", + "c-en-0012": "The method of claim 8 , wherein the match rule is specified by the user using Boolean logic.", + "c-en-0013": "The method of claim 8 , where the encryption scheme is a deterministic encryption scheme.", + "c-en-0014": "The method of claim 8 , wherein the encryption scheme is a probabilistic encryption scheme.", + "c-en-0015": "A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: receiving a match rule from a user of a customer relationship management tool, wherein the match rule comprises a unique identifier, a table in the customer relationship management tool, and one or more fields of applicability; and creating a match index column in the table comprising encrypted match indexes using the unique identifier as an initialization vector in an encryption scheme, wherein the match index column is used when performing duplicate detection to determine if a new record duplicates an encrypted match index in the encrypted match indexes.", + "c-en-0016": "The non-transitory computer-readable device of claim 15 , the operations further comprising: in response to receiving the new record for the table, calculating a match index for the new record based on the one or more fields of applicability; and when encryption is enabled on any of the one or more fields of applicability, encrypting the match index for the new record using the encryption scheme to derive a new encrypted match index.", + "c-en-0017": "The non-transitory computer-readable device of claim 16 , the operations further comprising: comparing the new encrypted match index to the encrypted match indexes in the match index column; and when no duplicate is found, adding the new record to the table.", + "c-en-0018": "The non-transitory computer-readable device of claim 16 , the operations further comprising: comparing the new encrypted match index to the encrypted match indexes in the match index column; and when a duplicate is found, displaying an error message.", + "c-en-0019": "The non-transitory computer-readable device of claim 15 , wherein the match rule is specified by the user using Boolean logic.", + "c-en-0020": "The non-transitory computer-readable device of claim 15 , where the encryption scheme is a deterministic encryption scheme or a probabilistic encryption scheme.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is continuation of U.S. application Ser. No. 16/026,819, now U.S. Pat. No. 10,942,906 by Hersans, et al., titled \u201cDetect Duplicates with Exact and Fuzzy Matching on Encrypted Match Indexes,\u201d which claims the benefit of U.S. Provisional Patent Application 62/678,812 by Hersans, et al., \u201cDetect Duplicates with Exact and Fuzzy Matching on Encrypted Match Indexes,\u201d filed May 31, 2018, both of which are incorporated by reference herein in their entireties.", + "p0003": "BACKGROUND", + "p0004": "Generally speaking, an organization or individual may utilize a cloud computing platform to manage relationships with customers. Such a cloud computing platform may be referred to as a customer relationship management solution. A CRM solution may include a wide-range of features such as contact management, sales management, and productivity tools to better track and analyze interactions with customers and potential customers. A CRM solution may accumulate large amounts of data to support these features.", + "p0005": "Keeping this data clean, up-to-date, and free from duplicates optimizes and enhances the performance and analytical utility of a CRM solution. However, cleaning the data may present challenges. For example, for purposes of data security, where customers need to encrypt data with an encryption key they control, CRM solutions may encrypt particular data fields or entities at-rest with the customer's encryption key. Such tenant-level encryption may further complicate duplicate detection and elimination in a CRM solution.", + "p0006": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0007": "The accompanying drawings are incorporated herein and form a part of the specification.", + "p0008": "FIG. 1 is a block diagram of a cloud computing system, according to some embodiments.", + "p0009": "FIG. 2 reflects a screenshot of a detected duplicate in a cloud computing platform, according to some embodiments.", + "p0010": "FIG. 3 reflects a screenshot of an encryption configuration screen for a standard field in a cloud computing platform, according to some embodiments.", + "p0011": "FIG. 4 reflects a screenshot of an encryption configuration screen for a custom field in a cloud computing platform, according to some embodiments.", + "p0012": "FIG. 5 reflects a screenshot of match rule in a cloud computing platform, according to some embodiments.", + "p0013": "FIG. 6 is a flowchart illustrating a method of duplicate detection when adding a new record in a cloud computing platform, according to some embodiments.", + "p0014": "FIG. 7 is a flowchart illustrating a method of creating and encrypting a match index to be used in duplicate detection, according to some embodiments.", + "p0015": "FIG. 8 is an example computer system useful for implementing various embodiments.", + "p0016": "DETAILED DESCRIPTION", + "p0017": "In the drawings, like reference numbers generally indicate identical or similar elements. Additionally, generally, the left-most digit of a reference number identifies the drawing in which the reference number first appears.", + "p0018": "DETAILED DESCRIPTION", + "p0019": "The following Detailed Description refers to accompanying drawings to illustrate exemplary embodiments consistent with the disclosure. References in the Detailed Description to \u201cone exemplary embodiment,\u201d \u201can exemplary embodiment,\u201d \u201can example exemplary embodiment,\u201d etc., indicate that the exemplary embodiment described may include a particular feature, structure, or characteristic, but every exemplary embodiment does not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases do not necessarily refer to the same exemplary embodiment. Further, when the disclosure describes a particular feature, structure, or characteristic in connection with an exemplary embodiment, those skilled in the relevant arts will know how to affect such feature, structure, or characteristic in connection with other exemplary embodiments, whether or not explicitly described.", + "p0020": "The exemplary embodiments described herein provide illustrative examples and are not limiting. Other exemplary embodiments are possible, and modifications may be made to the exemplary embodiments within the spirit and scope of the disclosure. Therefore, the Detailed Description does not limit the disclosure. Rather, the below claims and their equivalents define the scope of the disclosure.", + "p0021": "Hardware , firmware, software, or any combination thereof may be used to achieve the embodiments. Embodiments may also be implemented as instructions stored on a machine-readable medium and read and executed by one or more processors. A machine-readable medium includes any mechanism for storing or transmitting information in a form readable by a machine . For example, in some embodiments a machine-readable medium includes read-only memory ; random-access memory ; magnetic disk storage media; optical storage media; flash memory devices; electrical, optical, acoustical or other forms of propagated signals , and others. Further, firmware, software, routines, and/or instructions may be described herein as performing certain actions. However, these descriptions are merely for convenience, and these actions result from computing devices, processors, controllers, or other devices executing the firmware, software, routines, and/or instructions. Further, any implementation variations may be carried out by a general purpose computer, as described below.", + "p0022": "Any reference to the term \u201cmodule\u201d shall be understood to include at least one of software, firmware, and hardware or any combination thereof. In addition, those skilled in relevant arts will understand that each module may include one, or more than one, component within an actual device, and each component that forms a part of the described module may function either cooperatively or independently of any other component forming a part of the module. Conversely, multiple modules described herein may represent a single component within an actual device. Further, components within a module may be in a single device or distributed among multiple devices in a wired or wireless manner.", + "p0023": "The following Detailed Description of the exemplary embodiments will fully reveal the general nature of the disclosure so that others can, by applying knowledge of those skilled in relevant arts, readily modify and/or customize for various applications such exemplary embodiments, without undue experimentation and without departing from the spirit and scope of the disclosure. Therefore, such modifications fall within the meaning and plurality of equivalents of the exemplary embodiments based upon the teaching and guidance presented herein. Here, the phraseology or terminology serves the purpose of description, not limitation, such that the terminology or phraseology of the present specification should be interpreted by those skilled in relevant arts in light of the teachings herein.", + "p0024": "Provided herein are system, apparatus, device, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for enabling duplicate detection using encrypted match indexes in a cloud computing platform.", + "p0025": "Organizations may utilize a cloud computing platform as a CRM solution to manage relationships with customers. The cloud computing platform may allow the organization to track and analyze interactions with customers, increase sales, manage contacts, and better plan for the future. The cloud computing platform may organize customer service flows and supply-chain management and may monitor social media streams to determine potential customers. By tracking interactions with customers through such a cloud computing platform, organizations may ultimately improve profitability, eliminate process inefficiencies, and/or otherwise improve organizational competence.", + "p0026": "The cloud computing platform may store assorted and multitudinous data fields related to the organization, sales, customers, suppliers, competitors, leads, etc. Just for example, the cloud computing platform may store fields pertinent to contact information, customer preferences, social media data, customer purchase records, service records, customer interactions, marketing campaigns, sales targets, organizational objectives, sales data, profitability analyses, sales leads/opportunities, etc. Fields may be standard fields, e.g., contacts, accounts, leads, and opportunities, or custom fields designed and utilized by organizations for a particularized purpose tailored to their own organizational requirements.", + "p0027": "Because of the potentially sensitive nature of this data, a cloud computing platform may support data encryption. Encryption uses a key and an initialization vector while translating the otherwise readable alphanumeric data of a field into an encrypted form that is not readable to entities not privy to the key and the initialization vector. The cloud computing platform then stores the encrypted form of the ciphertext. When retrieving the stored data, the cloud computing platform may de-encrypt the ciphertext using the key and the initialization vector.", + "p0028": "Both standard and custom fields may be encrypted. Organizations may choose between encryption methodologies, e.g., deterministic and/or probabilistic, to encrypt a data field. A deterministic encryption scheme always produces the same ciphertext for a given plaintext and key. A probabilistic encryption scheme utilizes randomness in the encryption process, resulting in a different ciphertext given the same plaintext and key.", + "p0029": "Organizations also have an interest in maintaining clean data in a cloud computing platform. Clean data improves usability, prevents errors, maintains system integrity, and enhances analytical capabilities. One facet of maintaining clean data is the elimination of duplicates in a given data set. Duplicate detection may be managed globally by running duplicate elimination jobs. Duplicate detection may also occur on a table-by-table or case-by-case basis. Duplicate detection may occur automatically when adding a new record to the cloud computing platform.", + "p0030": "A cloud computing platform may utilize match rules to detect duplicitous records. A match rule may examine a particular field, or several fields, in the cloud computing platform. A match rule may be standard or custom . A standard match rule may examine pre-determined fields for a given data entity to determine if a duplicate exists. For example, a standardized match rule on contacts in a cloud computing platform may examine FirstName, LastName, and Address. A custom match rule examines user-configured fields in a customized fashion. For instance, a custom, user-defined match rule on contacts may be configured to also consider a contact's Title. In this custom rule, a duplicate would only be determined if FirstName, LastName, Address, and Title match. A user may be able to write Boolean logic to specify a match rule or utilize another programmatic manner to specify a match rule.", + "p0031": "Duplicate detection may employ an exact matching scheme or a fuzzy matching scheme. In an exact match scheme, only exact matches in the fields will return a positive result, i.e. a match. For example, if duplicate detection examined a FullName field, \u201cJohn Smith\u201d and \u201cJohn Smith\u201d would match but \u201cJohn Smith\u201d and \u201cJon Smith\u201d would not. Fuzzy matching provides a method that allows non-exact matches to be positively identified as a duplicate. In the example above, \u201cJon Smith\u201d may be positively identified as a duplicate of \u201cJohn Smith\u201d using a fuzzy matching scheme. Example approaches to fuzzy matching include: Jaro-Winkler, Kullback-Liebler distance, name variant, keyboard distance, Metaphone 3, and syllable alignment.", + "p0032": "A cloud computing platform may further provide duplicate detection on fields encrypted in various fashions, including fields encrypted via the encryption schemes described above. In supporting duplicate detection of these fields, a cloud computing platform may utilize auxiliary stored entries, e.g., match indexes, to facilitate duplicate detection. To avoid storing the plaintext of otherwise encrypted fields, the auxiliary fields may also need to be encrypted. In some embodiments, enabling encryption on fields in a cloud computing platform may interfere with a duplicate detection system that examines those fields. In one non-limiting example, a cloud computing platform may store encrypted fields in a fashion only related to the original seed text through the use of a key and an initialization vector. In such an example, conventional duplicate detection may not work on the encrypted fields because a simple comparison between a match rule composed of unencrypted keys and the encrypted field will not result in a positive identification of a duplicate. In another embodiment, utilizing probabilistic encryption may result in duplicate detection being disabled due to the varying and unpredictable results of a probabilistic encryption scheme. Accordingly, a need exists to allow a cloud computing platform to detect duplicates in encrypted fields by utilizing encrypted match indexes.", + "p0033": "FIG. 1 is a block diagram of a system 100 , according to some embodiments. Cloud computing system 100 may include cloud computing platform 102 , user system 104 , network 106 , host application 108 , record store 110 , encryption engine 112 , and matching engine 114 . System 100 may connect cloud computing platform 102 to user system 104 via network 106 . As would be appreciated by a person of ordinary skill in the art , cloud computing platform 102 may be connected to multiple user systems 104 .", + "p0034": "Cloud computing platform 102 may be a server computer, desktop computer, laptop, tablet, mobile device, wearable electronic device, or other electronic device. Cloud computing platform 102 may also be a software platform for cloud computing. For example, cloud computing platform 102 may be Software as a Service cloud platform where a user subscribes to an application accessible via the Internet. Cloud computing platform 102 may provide CRM-related services. Such services may allow organizations to track and analyze interactions with customers. Cloud computing platform 102 may provide a litany of other functionalities.", + "p0035": "User system 104 may be a person interacting with cloud computing platform 102 . User system 104 may be a cellphone, smartphone, tablet computer, laptop computer, desktop computer, web browser, or any other suitable computing device. User system 104 may be configured to connect to and communicate with cloud computing platform 102 . A user of user system 104 may be a business owner, employee, agent, or another suitable individual interacting with information relevant to a business, company, non-profit, governmental agency, or any other suitable organization. Conversely, a user of user system 104 may use cloud computing platform 102 for individual pursuits or reasons unrelated to any business or organizational goals.", + "p0036": "Network 106 may be any network or combination of networks including the Internet, a local area network , a wide area network , a wireless network, a cellular network, or various other types of networks as would be appreciated by a person of ordinary skill in the art.", + "p0037": "Host application 108 may be included in cloud computing platform 102 and may be any suitable type of application. For example, host application 108 may be a CRM application currently running on cloud computing platform 102 . In some embodiments, host application 108 may be a web application designed to run inside a web browser. In some other embodiments, host application 108 may be a software application designed to run on a server computer, desktop computer, or other type of electronic device. Host application 108 may be designed and deployed for running on a mobile platform. Host application 108 may run in any suitable operating system environment. As would be appreciated by a person of ordinary skill in the art , host application 108 may be another type of software application.", + "p0038": "Record store 110 may be a database or other type of data store. A data object may be an item or collection of data. Record store 110 may store fields related to the organization, sales, customers, suppliers, competitors, leads, etc. The fields stored by record store 110 may be standard fields or custom fields. Record store 110 may also store match indexes to be utilized in detecting duplicates. Record store 110 may support encryption of the data fields contained in record store 110 .", + "p0039": "Encryption engine 112 may be utilized by cloud computing platform 102 to encrypt fields in record store 110 with a deterministic, probabilistic, or other suitable encryption scheme. Encryption engine 112 may be employed by cloud computing platform 102 or host application 108 . Encryption engine 112 may encrypt fields in record store 110 . Encryption engine 112 may utilize a suitable seed, key, or initialization vector to encrypt the fields in record store 110 . Encryption engine 112 may also encrypt match indexes used to detect duplicates in cloud computing platform 102 . Encryption engine 112 may retrieve the unique identifier stored in record store 110 for a match rule and use the unique identifier as the key or initialization vector when encrypting a record in record store 110 . Encryption engine 112 may subsequently de-encrypt any stored encrypted values for utilization. Encryption engine 112 may utilize or adhere to an appropriate encryption standard or specification such as 2TDEA, 3TDEA, AES-128, AES-192, AES-256, etc.", + "p0040": "Matching engine 114 may determine duplicates stored in record store 110 . Matching engine 114 may also check new data received by host application 108 prior to inserting the data into record store 110 to ensure that a duplicate will not be introduced by the insertion. Matching engine 114 may be employed by cloud computing platform 102 or host application 108 . Matching engine 114 may determine if duplicates presently exist in record store 110 for unencrypted fields by comparing unencrypted fields to a match index created for each record in a data entity. In one embodiment, matching engine 114 may perform a two-phase match wherein the first phase performs exact matching and the second phase performs appropriate fuzzy matching algorithms to determine duplicates. Matching engine 114 may further determine if duplicates exist in record store 110 for encrypted fields, as described in further detail below with reference to FIG. 3 and FIG. 4 .", + "p0041": "FIG. 2 reflects a screenshot of a detected duplicate in cloud computing platform 102 . This is just an exemplary embodiment. FIG. 2 portrays an exemplary interface that includes account 202 , new contact button 204 , duplicate notification 206 , and account details 208 . FIG. 2 displays account 202 , but this could be another standard or custom field within cloud computing platform 102 . New contact button 204 is an exemplary input field that may receive input from a user allowing the user to enter updated information for fields such as contacts, accounts, etc. Duplicate notification 206 may display when the system detects a duplicate through an appropriate method, as elaborated on further in the discussions of FIG. 6 and FIG. 7 below. In this screenshot, solely for example, a match rule may have been created on account name, phone, and owner. Duplicate notification 206 may here display because another record stored in record store 110 may an account name of \u201cGlobal Media,\u201d a phone number of \u201c 555-1212,\u201d and an owner of \u201cYves Martin.\u201d Account details 208 may display various fields pertinent to the standard of custom data field being displayed.", + "p0042": "FIG. 3 reflects a screenshot of an encryption configuration screen for a standard field in cloud computing platform 102 . This is just an exemplary embodiment. FIG. 3 portrays an exemplary interface that includes account box 302 , checkbox 304 , field label 306 , encryption settings 308 , and type selector 310 . FIG. 3 displays an account box 302 but this may be another standard within cloud computing platform 102 , e.g., contacts, sales, etc. Checkbox 304 provide a mechanism by which a user may enable encryption across the fields relevant to this exemplary account. Enabling encryption on a standard field may result in encrypting the data as stored in record store 112 as described below with reference to FIG. 6 and FIG. 7 . Field label 306 are the labels to the fields relevant to this exemplary. Encryption settings 308 provide a framework for adjusting type selector 310 . In this embodiment, type selector 310 indicate either probabilistic or deterministic encryption. One skilled in the arts would appreciate that the fields in account box 302 may vary based on the type of standard data.", + "p0043": "FIG. 4 reflects a screenshot of an encryption configuration screen for a custom field in cloud computing platform 102 . This is just an exemplary embodiment. FIG. 4 portrays an exemplary interface that includes field label 402 , field name 404 , description 406 , help text 408 , encryption enabler 410 , and type selector 412 . Field label 402 , field name 404 , description 406 , and help text 408 may specify details about a custom field in cloud computing platform 102 . Utilizing encryption enable 410 , a user may enable encryption on the custom field in cloud computing platform 102 . The type of encryption, e.g., deterministic vs. probabilistic may be toggled using type selector 412 . Enabling encryption on a custom field may result in encrypting the data as stored in record store 112 as described below with reference to FIG. 6 and FIG. 7 .", + "p0044": "FIG. 5 reflects a screenshot of match rule creation screen in cloud computing platform 102 . This is just an exemplary embodiment. FIG. 5 portrays an exemplary interface that includes standard selector 502 , standard rule selector 504 , custom selector 506 , custom rule selector 508 , criteria string 510 , add button 512 , and remove button 514 . As discussed below with reference to FIG. 7 , a match rule may be standard or user-defined. Standard selector 502 and standard rule selector 504 may portray a standard match rule configuration. Custom selector 506 , custom rule selector 508 , and criteria string 510 may portray a custom match rule. Add rule button 512 and remove button 512 may allow a user to add or remove a match rule. The utilization of match rules in the determination of duplicates in a cloud computing platform is described in further detail below with reference to FIG. 6 .", + "p0045": "FIG. 6 is a flowchart illustrating a method 600 of duplicate detection when adding a new record in a cloud computing platform, according to some embodiments. Method 600 can be performed by processing logic that can comprise hardware , software , or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 6 , as will be understood by a person of ordinary skill in the art. Method 600 shall be described with reference to FIG. 1 . However, method 600 is not limited to that example embodiment.", + "p0046": "In 602 , host application 108 may receive a new record from user system 104 . The new record may represent a new standard field such as a contact, account, lead, or opportunity. The new record may also represent a new custom field previously configured within host application 108 via a user of user system 104 . The new record may be received using appropriate HTML form data, in XML, or any other suitable information transfer format. In an illustrative example, host application 108 may receive a new Contact comprising a first name , a last name , a mailing address , a title , and a contact number . This disclosure will utilize this example in the discussion of the subsequent steps that follows. In an embodiment, such a contact may also be received through an automated process, for example, an import mechanism or other automated propagation of data.", + "p0047": "In 604 , host application 108 retrieves stored match rules pertinent to the new record received in 602 . In the above example, host application 108 may retrieve from record store 110 any match rules configured to detect duplicates on the Contacts field. In an alternate example, host application 108 may receive match rules for a custom data entity. One skilled in the art will understand that more than one match rule may be applicable to a given data entity, depending on configurations stored in record store 110 . Thus, host application 108 may retrieve more than one match rule, and in this embodiment, the subsequent steps may be iterative. To continue the above contact example, host application 108 may retrieve a match rule on FirstName, LastName, Address, and Title. Host application may retrieve the fields to which the match rule applies, the type of matching to perform for those fields , and other appropriate configuration information. The match rule may also include a unique identifier stored in record store 110 to identify that particular match rule.", + "p0048": "In 606 , host application 108 calculates a match index for the new record. The nature of the match index may vary based on the nature of the data being received. Host application 108 may derive a different match index corresponding to each match rule received in 604 if more than one match rule is in place. The match index may be a composite of the data fields received. For the above example, the match index across FirstName, LastName, Address, and Title may be determined to be \u201cJSmithMapleDeveloper.\u201d This example is in no way limiting, however, and a multitude of varied approaches may be taken to determine the match index; those approaches may vary based on the nature of the data entity at issue.", + "p0049": "In 608 , host application 108 determines if encryption is enabled on any of the fields comprising the match index. In the above example, host application 108 may determine from record store 110 if any one of first name, last name, mailing address, title, and a contact number is encrypted. In an embodiment, host application 108 may behave differently if the encryption scheme is probabilistic versus deterministic. In one embodiment, duplicate determination may not function if the encryption scheme is probabilistic. If encryption is enabled for any of the fields, then method 600 moves to 612 . If encryption is not enabled for any of the fields, then method 600 moves to 610 .", + "p0050": "In 610 , host application 108 compares the match index determined in 606 to the rows in the relevant data entity in record store 110 . Because encryption is not enabled on the fields comprising the match index, records store 110 may store the match index in an unencrypted form. Thus for the above example, hosting application 108 may perform a table scan to determine if \u201cJSmithMapleDeveloper\u201d exists in the relevant data entity in record store 110 . Host application 108 may perform any other suitable search mechanism to determine if the match index is duplicative.", + "p0051": "In 612 , host application 108 encrypts the match index determined in 606 via encryption engine 112 . Host application 108 may utilize the unique identifier for the match rule as the initialization vector that keys the encryption scheme. Thus, encryption engine 112 may derive appropriate ciphertext from the \u201cJSmithMapleDeveloper\u201d plaintext. The ciphertext may consist of alphanumeric characters and may be a configurable bit-length.", + "p0052": "In 614 , host application 108 compares the encrypted match index determined in 612 to the encrypted match index column in the relevant data entity in record store 110 . For the above example, hosting application 108 may perform a table scan to determine if the encrypted ciphertext exists in the relevant data entity in record store 110 . Host application 108 may perform any other suitable search mechanism to determine if the encrypted match index is duplicative.", + "p0053": "In 616 , host application 108 determines if a duplicate was found in step 610 or 614 . If a duplicate record was found, then method 600 proceeds to 618 . If a duplicate record was not found, then method 600 proceeds to 620 .", + "p0054": "In 618 , host application 108 returns an error message indicating that a duplicate record was found. In an embodiment, user system 104 may be asked to confirm the information or update the information if appropriate.", + "p0055": "In 620 , host application 108 adds the new record to record store 110 . The record may include a column housing the match index. If encryption is not enabled for the relevant fields, the match index may be stored in this column. If encryption is enabled, the encrypted match index may be stored in this column.", + "p0056": "FIG. 7 is a flowchart illustrating a method 700 of creating and encrypting a match rule to be used in duplicate detection, according to some embodiments. Method 700 can be performed by processing logic that can comprise hardware , software , or a combination thereof. It is to be appreciated that not all steps may be needed to perform the disclosure provided herein. Further, some of the steps may be performed simultaneously, or in a different order than shown in FIG. 7 , as will be understood by a person of ordinary skill in the art. Method 700 shall be described with reference to FIG. 1 . However, method 700 is not limited to that example embodiment.", + "p0057": "In 702 , host application 108 receives match rule parameters from a user of user system 704 . The parameters may specify the data entity to which the match rule applies, the fields in the data entity to which the match rule applies, Boolean or other logic specifying how the match rule should behave, whether blanks or null values should be treated as a match, and other suitable configuration information. The match rule parameters may be received using appropriate HTML form data, in XML, or any other suitable information transfer format. In one embodiment, host application could receive match rule parameters updating an existing rule.", + "p0058": "In 704 , host application 108 may create the match rule in record store 110 . Host application may allow more than one match rule to examine the same fields, however, suitable redundancy checks may be performed. Appropriate rows and columns may be added to particular tables in order to track the match rule. When modifying an existing match rule, host application 108 may perform updates to the underlying table as opposed to inserts.", + "p0059": "In 706 , host application 108 may create a match index column based on the entries currently existing in the specified data entity. For example, if the match rule examines FirstName, LastName, and Title, then host application 108 may create an appropriate match index for each record in the data entity, i.e., each row in the table, based on the data contained in the table. Host application 108 may perform this row by row, in batches, iteratively, recursively, or in any other suitable programmatic fashion. Host application 108 may store the match indexes for each row temporarily in memory, text, temporary database tables, or any other suitable storage medium.", + "p0060": "In 708 , host application 108 determines if encryption is enabled on any of the fields examined by the match rule. Host application 108 may retrieve data from record store 110 about the fields and data entities for which encryption is enabled. Host application 108 may also fetch information about the type of encryption applied to the data entities. In an embodiment, host application 108 may behave differently based on the type of encryption. If encryption is enabled, then method 700 proceeds to 710 . If encryption is not enabled, then method 700 proceeds to 714 .", + "p0061": "In 710 , where encryption is not enabled or needed to store the match index column, host application 108 may store the match index column in record store 110 in unencrypted form.", + "p0062": "In 712 , host application 108 may encrypt the entirety of the match index column created in 706 . Host application 108 may utilize a unique identifier for the match rule as the initialization vector for the encryption scheme. Host application 108 may retrieve the match indexes created in 706 from memory, text, temporary database tables, etc. Host application 108 may encrypt the entries in the match index column one by one, in batches, or in their entirety.", + "p0063": "In 714 , host application 108 stores the encrypted form of the match index column in record store 110 . Host application 108 may store the encrypted match indexes in the column through bulk inserts, database updates, text manipulation, or other suitable storage approach.", + "p0064": "Various embodiments can be implemented, for example, using one or more computer systems, such as computer system 800 shown in FIG. 8 . Computer system 800 can be used, for example, to implement method 600 of FIG. 6 and method 700 of FIG. 7 . For example, computer system 800 can an initialization of an analytics system using pre-set global filters or update. Computer system 800 can be any computer capable of performing the functions described herein.", + "p0065": "Computer system 800 can be any well-known computer capable of performing the functions described herein.", + "p0066": "Computer system 800 includes one or more processors , such as a processor 804 . Processor 804 is connected to a communication infrastructure or bus 806 .", + "p0067": "One or more processors 804 may each be a graphics processing unit . In an embodiment, a GPU is a processor that is a specialized electronic circuit designed to process mathematically intensive applications. The GPU may have a parallel structure that is efficient for parallel processing of large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.", + "p0068": "Computer system 800 also includes user input/output device 803 , such as monitors, keyboards, pointing devices, etc., that communicate with communication infrastructure 806 through user input/output interface 802 .", + "p0069": "Computer system 800 also includes a main or primary memory 808 , such as random access memory . Main memory 808 may include one or more levels of cache. Main memory 808 has stored therein control logic and/or data.", + "p0070": "Computer system 800 may also include one or more secondary storage devices or memory 810 . Secondary memory 810 may include, for example, a hard disk drive 812 and/or a removable storage device or drive 814 . Removable storage drive 814 may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.", + "p0071": "Removable storage drive 814 may interact with a removable storage unit 818 . Removable storage unit 818 includes a computer usable or readable storage device having stored thereon computer software and/or data. Removable storage unit 818 may be a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, and/any other computer data storage device. Removable storage drive 814 reads from and/or writes to removable storage unit 818 in a well-known manner.", + "p0072": "According to an exemplary embodiment, secondary memory 810 may include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system 800 . Such means, instrumentalities or other approaches may include, for example, a removable storage unit 822 and an interface 820 . Examples of the removable storage unit 822 and the interface 820 may include a program cartridge and cartridge interface , a removable memory chip and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.", + "p0073": "Computer system 800 may further include a communication or network interface 824 . Communication interface 824 enables computer system 800 to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. . For example, communication interface 824 may allow computer system 800 to communicate with remote devices 828 over communications path 826 , which may be wired and/or wireless, and which may include any combination of LANs, WANs, the Internet, etc. Control logic and/or data may be transmitted to and from computer system 800 via communication path 826 .", + "p0074": "In an embodiment, a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, computer system 800 , main memory 808 , secondary memory 810 , and removable storage units 818 and 822 , as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices , causes such data processing devices to operate as described herein.", + "p0075": "Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art how to make and use embodiments of this disclosure using data processing devices, computer systems and/or computer architectures other than that shown in FIG. 8 . In particular, embodiments can operate with software, hardware, and/or operating system implementations other than those described herein.", + "p0076": "It is to be appreciated that the Detailed Description section, and not any other section, is intended to be used to interpret the claims. Other sections can set forth one or more but not all exemplary embodiments as contemplated by the inventor , and thus, are not intended to limit this disclosure or the appended claims in any way.", + "p0077": "The breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.", + "features": { + "F1": "Receiving, by one or more processors, a match rule from a user of a customer relationship management tool, wherein the match rule comprises a unique identifier, a table in the customer relationship management tool, and one or more fields of applicability.", + "F2": "Creating, by the one or more processors, a match index column in the table comprising encrypted match indexes using the unique identifier as an initialization vector in an encryption scheme.", + "F3": "The match index column is used when performing duplicate detection to determine if a new record duplicates an encrypted match index in the encrypted match indexes.", + "F4": "Calculating, by the one or more processors, a match index for a new record based on the one or more fields of applicability in response to receiving the new record for the table.", + "F5": "Encrypting the match index for the new record using the encryption scheme to derive a new encrypted match index when encryption is enabled on any of the one or more fields of applicability.", + "F6": "Comparing, by the one or more processors, the new encrypted match index to the encrypted match indexes in the match index column; and when no duplicate is found, adding the new record to the table.", + "F7": "Comparing, by the one or more processors, the new encrypted match index to the encrypted match indexes in the match index column; and when a duplicate is found, displaying an error message.", + "F8": "The match rule is specified by the user using Boolean logic.", + "F9": "The encryption scheme is a deterministic encryption scheme.", + "F10": "The encryption scheme is a probabilistic encryption scheme.", + "F11": "A system comprising a memory and at least one processor coupled to the memory configured to perform the operations described in F1 to F5.", + "F12": "A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the device to perform operations comprising the features described in F1 to F5." + } + } + }, + { + "FAN": "79045164", + "CPC": [ + "B23Q-017/00", + "G05B-019/042/8", + "G05B-019/4063", + "G05B-019/409", + "G05B-023/02", + "G05B-2219/23157", + "G05B-2219/35494", + "G06F-003/147", + "G06F-016/00", + "G06Q-010/00" + ], + "Content": { + "title": "Electronic manual for machine tool and machine tool", + "pa01": "An electronic manual for a machine tool for displaying information related to operation, maintenance, or inspection of the machine tool, which is necessary for an operator, includes an operator-operated input part for inputting search conditions to search the electronic manual in accordance with intent of the operator in order to obtain necessary information, a machine-operated input part for selecting and inputting at least one item provided to the operator from the machine tool depending on a status of the machine tool, and a display part for displaying information related to operation, maintenance, or inspection of the machine tool corresponding to the input from the operator-operated input part or the machine-operated input part.", + "c-en-0001": "An electronic manual for a machine tool for displaying information related to operation, maintenance, or inspection of the machine tool, which is necessary for an operator, the electronic manual comprising: a display; and a controller for the display, wherein the controller is configured to count elapsed time since prior inspection for one or more inspection items, and wherein the controller is configured to control the display so as to display: a first window, including a list of units constituting the machine tool, and allowing an operator to select one of the units, a second window, when one of the units is selected, the second window including a plurality of items related to operation, maintenance or inspection of the selected unit, and allowing the operator to select and input one of the items to search the electronic manual in accordance with intent of the operator in order to obtain necessary information, a third window, without being prompted by the operator, when the counted elapsed time reaches a predetermined time set for each of the inspection items, the third window allowing the operator to select and input at least one of the inspection items, in order to obtain necessary information related thereto, and information related to operation, maintenance, or inspection of the machine tool corresponding to the input from the second window or the third window.", + "c-en-0002": "The electronic manual for a machine tool according to claim 1 , wherein the electronic manual is stored in a controller of the machine tool, an Internet-connected server, or a mobile terminal.", + "c-en-0003": "The electronic manual for a machine tool according to claim 2 , wherein a two-dimensional code related to information related to operation, maintenance, or inspection of the machine tool displayed on the display part is further displayed on the display part, and the mobile terminal is connected to a server via the Internet as a result of reading of the two-dimensional code by the mobile terminal, and the information related to operation, maintenance, or inspection of the machine tool displayed on the display part is displayed on the mobile terminal.", + "c-en-0004": "A machine tool in which a workpiece and a tool are moved relative to each other to machine the workpiece, the machine tool comprising: a table on which the workpiece is placed; a tool for machining the workpiece; a display; and a controller for the machine tool, wherein the controller is configured to count elapsed time since prior inspection for each inspection items, and wherein the controller is configured to control the display so as to display: a first window, including a list of units constituting the machine tool, and allowing an operator to select one of the units, a second window, when one of the units is selected, the second window including a plurality of items related to operation, maintenance or inspection of the selected unit, and allowing the operator to select and input one of the items to search the electronic manual in accordance with intent of the operator in order to obtain necessary information, a third window, without being prompted by the operator, when the counted elapsed time reaches a predetermined time set for each of the inspection items, the third window allowing the operator to select and input at least one of the inspection items, in order to obtain necessary information related thereto, and information related to operation, maintenance, or inspection of the machine tool corresponding to the input from the second window or the third window.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a U.S. national phase patent application of International Patent Application No. PCT/JP2016/076701, filed Sep. 9, 2016, which is hereby incorporated by reference in the present disclosure in its entirety.", + "p0003": "FIELD OF THE INVENTION", + "p0004": "The present invention relates to an electronic manual for a machine tool for displaying information related to the operation, maintenance, or inspection of the machine tool, which is necessary for an operator, and a machine tool which can display the electronic manual.", + "p0005": "BACKGROUND OF THE INVENTION", + "p0006": "Since machine tools have a wide variety of specifications, mechanisms, functions, and target groups which are complicated, the instruction manuals therefor become enormous. A certain degree of skill is necessary in order to find necessary information in such an enormous instruction manual, and some users cannot obtain necessary information even while in possession of the instruction manual. Furthermore, machine tools are generally used for long periods of time, during which parts are improved on the manufacturer side and new products can be used for the lubricating oils and coolants thereof, which are consumables.", + "p0007": "Patent Literature 1 describes a machining device in which an input screen for inputting operation items is displayed on a display monitor of the machining device by means of the reading of identification codes recorded in an instruction manual with a reader connected to the controller of the machining device.", + "p0008": "PATENT LITERATURE", + "p0009": "Japanese Unexamined Patent Publication No. 2014-24150", + "p0010": "BRIEF SUMMARY OF THE INVENTION", + "p0011": "In the machining device of Patent Literature 1, the time and effort of the inputting of maintenance information are reduced, and maintenance information can be accurately inputted in a short time. However, there are various types of maintenance, and the individual maintenance operations have different operation methods and time intervals at which the maintenance should be performed. Simply by simplifying the input operation as in Patent Literature 1, it remains unclear which operation item it is necessary to input in order to obtain appropriate maintenance information. Furthermore, in the case of not only maintenance and inspection but also malfunctions in the machining device, it is difficult to obtain information related to the operation to resolve the malfunction unless the correspondence between the malfunction and the operation item to be input to the display monitor is known.", + "p0012": "The present invention aims to solve such problems of the prior art by providing an electronic manual for a machine tool which can display information related to the operation, maintenance, and inspection of a machine tool, depending on a status, in a simple manner, and a machine tool which can display such an electronic manual.", + "p0013": "In order to achieve the above object, according to the present invention, there is provided an electronic manual for a machine tool for displaying information related to operation, maintenance, or inspection of the machine tool, which is necessary for an operator, the electronic manual comprising an operator-operated input part for inputting search conditions to search the electronic manual in accordance with intent of the operator in order to obtain necessary information, a machine-operated input part for selecting and inputting at least one item provided to the operator from the machine tool depending on a status of the machine tool, and a display part for displaying information related to operation, maintenance, or inspection of the machine tool corresponding to the input from the operator-operated input part or the machine-operated input part.", + "p0014": "Further, according to the present invention, there is provided a machine tool in which a workpiece and a tool are moved relative to each other to machine the workpiece, the machine tool comprising an operator-operated input part for inputting search conditions to search an electronic manual in accordance with intent of the operator in order to obtain necessary information, a machine-operated input part for selecting and inputting at least one item provided to the operator from the machine tool depending on a status of the machine tool, and a display part for displaying information related to operation, maintenance, or inspection of the machine tool corresponding to the input from the operator-operated input part or the machine-operated input part.", + "p0015": "According to the present invention, necessary information can be located in a simple manner even when the operator lacks skill in relation to the searching of the instruction manual. In particular, at least one item is provided from the machine tool depending on the status of the machine tool, and the operator can perform input by selecting the item, whereby information depending on the status of the machine tool can be reliably provided to the operator.", + "p0016": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0017": "FIG. 1 is a side view of a machine tool according to an embodiment of the present invention.", + "p0018": "FIG. 2 is a block diagram showing an electronic manual which is connected to a server via the Internet.", + "p0019": "FIG. 3 is a view showing an example of an alarm window constituting a machine-operated input part displayed on a display part.", + "p0020": "FIG. 4 is a view showing an example of a periodic inspection window constituting the machine-operated input part displayed on the display part.", + "p0021": "FIG. 5 is a view of the periodic inspection window of FIG. 3 , which shows a two-dimensional code.", + "p0022": "FIG. 6 is a view showing an example of a search window constituting an operator-operated input part displayed on the display part.", + "p0023": "FIG. 7 is a view showing an example of a search method selection window constituting the operator-operated input part displayed on the display part.", + "p0024": "FIG. 8 is a view showing an example of a unit search window constituting the operator-operated input part displayed on the display part.", + "p0025": "FIG. 9 is a view of the unit search window of FIG. 7 in which one item is open.", + "p0026": "FIG. 10 is a view of the unit search window of FIG. 7 in which two items are open.", + "p0027": "DETAILED DESCRIPTION OF THE INVENTION", + "p0028": "Preferred embodiments of the present invention will be described below with reference to the attached drawings.", + "p0029": "FIG. 1 shows an example of a machine tool to which the present invention is applied. In FIG. 1 , the machine tool 100 according to a preferred embodiment of the present invention constitutes a vertical machining center which is controlled by a controller 130 . The machine tool 100 comprises a bed 102 as a base fixed to the floor surface of a factory, a table 106 which is provided on an upper surface of a front portion of the bed 102 so as to be moveable in the forwards and backwards directions or the Y-axis directions and on which a workpiece W is fixed, an upright column 104 which is fixed on the upper surface of the bed 102 on the rear end side of the bed 102 , an X-axis slider 108 which is provided on a front surface of the column 104 so as to be movable in the left and right directions or the X-axis directions , and a spindle head 110 which is attached on a front surface of the X-axis slider 108 so as to be movable in the upward and downward directions or the Z-axis directions and which rotatably supports a spindle 112 . A tool 114 for machining the workpiece W fixed to the table 106 is mounted on the tip of the spindle 112 .", + "p0030": "The machine tool 100 further comprises an operation panel 120 with which an operator can operate the machine tool 100 . The operation panel 120 comprises a touch panel 122 serving as the display part with which a desired portion can be selected by touching a screen. The operation panel 120 is connected to the controller 130 in which the electronic manual is stored. As shown in FIG. 2 , the electronic manual may be stored on a server 162 and the operation panel 120 may be connected to the server via the Internet 160 . When connection is established via the Internet 160 , individual machine tools have security information for accessing the server such as accounts, passwords or user authentication information.", + "p0031": "In FIGS. 3 to 10 , when the operator operates the electronic manual, a main window 10 for displaying information related to the machine tool is displayed on the touch panel 122 of the operation panel 120 . The main window 10 includes an area 18 for displaying an alarm window 20 , a periodic inspection window 30 , and a search window 50 , which are described later, a selection button 12 , a two-dimensional code generation button 14 , and a search button 16 .", + "p0032": "The controller 130 of the machine tool 100 continuously monitors each part of the machine tool 100 when the machine tool 100 is powered on and generates an alarm when a malfunction occurs in any of the parts of the machine tool 100 . In particular, in the present embodiment, as shown in FIG. 2 , the alarm window 20 is displayed in the main window 10 on the touch panel 122 , whereby the operator can be notified that a malfunction has occurred in the machine tool 100 .", + "p0033": "Referring to FIG. 3 , the alarm window 20 includes an area 22 for displaying that the window is an alarm window, an area 24 for displaying an alarm code, and an area 26 for displaying a malfunction item. Table 1 shows examples of malfunction items along with alarm codes. Though a single malfunction item is displayed in FIG. 3 , a plurality of malfunction items can be displayed.", + "p0034": "TABLE 1 Alarm Code Item 60131 ATC Shutter Open Start Condition Abnormality 60132 ATC Shutter Close Start Condition Abnormality 60133 ATC Operation Start Condition Abnormality 60134 Spindle Tool Clamp/Unclamp Operation Start Condition Abnormality 60135 ATC Operation Start Condition Abnormality 60136 ATC Operation Condition Abnormality 60137 ATC Operation Condition Abnormality 60138 ATC Operation Condition Abnormality 70047 Spindle Command Abnormality 70048 Clamp Mechanism State Abnormality 70049 Spindle Start-Up Condition Abnormality 70050 Set Speed Abnormality 70051 Spindle Start-Up Condition Abnormality 70052 Spindle Start-Up Condition Abnormality 70053 Drawbar Probe Advancement Start Condition Abnormality 70054 Condition Abnormality During Drawbar Probe Advancement Operation 920033 Condition Abnormality During Pallet Change Operation 920034 Reconfirming Pallet Seating Confirmation 920035 Pallet Seating Confirmation Limit Switch Abnormality 920036 Pallet Seating Abnormality 920037 Pallet Seating Abnormality 920040 Condition Abnormality During Pallet Change Operation", + "p0035": "Furthermore, FIG. 4 shows a periodic inspection window displayed on the touch panel 122 of the operation panel 120 as a message issued from the controller 130 of the machine tool 100 to the operator in order to prompt the operator to perform a periodic inspection. The periodic inspection is an item performed regardless of the uptime of the machine, such as a six-month inspection, a one-year inspection, or a two-year inspection, or an item performed in accordance with the uptime of the machine, such as a 160-hour inspection, a 500-hour inspection, or a 1000-hour inspection. Table 2 shows examples of periodic inspections.", + "p0036": "TABLE 2 Please inspect oil amount in oil/air supply device and replenish if low Please clean spindle taper part Please inspect oil filter of oil controller and clean if very dirty Please inspect oil flow of oil controller with the flow gauge Please inspect spindle-through line filter/suction filter Please inspect drum filter backwash amount Please inspect waste oil tank Please inspect amount of cutting oil and replenish if low Please inspect scale air pressure Please inspect main air pressure Please inspect amount of hydraulic oil of hydraulic device Please inspect pressure of hydraulic oil of hydraulic device Please adjust tension of outboard conveyor chain Please replace filter net of drum filter Please clean/replace line filter/suction filter Please inspect filter net of drum filter Please inspect coolant V seal Please clean coolant tank Please replace scale air oil mist filter Please replace main air filter Please clean air dryer condenser fins Please clean suction filter of hydraulic device Please clean fan cooler of hydraulic device Please replace hydraulic oil of hydraulic device Please replenish grease of Y-axis pantograph", + "p0037": "Regarding each inspection item, the controller 130 of the machine tool 100 , simply counts the time which has elapsed since the prior inspection or since the performance of a maintenance operation associated with an inspection for items executed irrespective of uptime, and counts the elapsed time in accordance with the uptime of the machine regarding items executed in accordance with the uptime of the machine. When the counted elapsed time reaches a value specified for an inspection item, the controller 130 displays the periodic inspection window 30 on the touch panel 122 of the operation panel 120 . The periodic inspection window 30 includes an area 32 for displaying that the window is a periodic inspection window, and an area for displaying inspection items 34 to 38 .", + "p0038": "When the alarm window 20 or the periodic inspection window 30 is open, if the operator taps and selects one or a plurality of items from the malfunction item 34 of the inspection items 34 to 38 and taps the selection button 12 , information related to the selected item is displayed on the touch panel 122 . When a plurality of items are selected, each of the plurality of items can be displayed in an individual tab. The information displayed on the touch panel 122 includes the name of the part in which the malfunction has occurred, the method for accessing the part in which the malfunction has occurred, an alarm resolution procedure, the name, serial number, and replacement method of a part to be replaced, the method for accessing the part for which periodic inspection should be performed, inspection methods such as visual inspection and verification of gauges, a maintenance procedure, and the name, serial number, and replacement method of a part to be replaced.", + "p0039": "Furthermore, when the two-dimensional code generation button 14 is tapped after item selection, a two-dimensional code 40 is displayed on the touch panel 122 , as shown in FIG. 5 . As shown in FIG. 2 , when the two-dimensional code 40 is read by a mobile terminal 124 , information can be displayed on the mobile terminal 124 via the Internet 160 . The two-dimensional code 40 can include a URL and security information necessary for the mobile terminal to access the server on which the electronic manual is stored, as well as the above malfunction items and periodic inspection items. Furthermore, the mobile terminal 124 can be a commercially available smartphone or tablet, rather than a special device provided by the manufacturer of the machine tool 100 .", + "p0040": "Thus, when a malfunction occurs in a part of the machine tool 100 , or when the amount of time which has elapsed since the prior periodic inspection becomes a value specified for an inspection item, by displaying the alarm window 20 or the periodic inspection window 30 on the touch panel 122 of the operation panel 120 , the operator can be prompted by the machine tool 100 to perform an operation to resolve the malfunction or to perform the periodic inspection. In the present invention, the alarm window 20 and the periodic inspection window 30 constitute machine-operated input parts.", + "p0041": "When the alarm window 20 or the periodic inspection window 30 is displayed on the touch panel 122 , by tapping the alarm window 20 or the periodic inspection window 30 , information related to a resolution operation, such as an operation to resolve a malfunction, a method for accessing a part of the machine tool 100 for which an inspection operation should be performed, or a method for replacement of a part, or an inspection operation can be provided to the operator in a simple manner. Furthermore, by displaying the two-dimensional code 40 on the touch panel 122 and reading the two-dimensional code 40 with a smartphone or tablet, information related to the resolution operation or inspection operation is displayed on the smartphone or tablet, whereby information can be provided to the operator in a location separated from the touch panel 122 .", + "p0042": "In the embodiment described above, the controller 130 of the machine tool 100 automatically displays the alarm window 20 and the periodic inspection window 30 on the touch panel 122 . However, naturally, the operator can refer to the instruction manual as necessary. FIG. 6 shows a search window with which the operator can search the electronic manual in such a case.", + "p0043": "The search window 50 is opened by tapping the search button 16 of the main window 10 . The search window 50 includes a plurality of items corresponding to major items in the table of contents of a paper instruction manual. When the operator taps and selects one item from the plurality of items, a search method selection window 54 with which the operator can select a search method from a plurality of search methods for searching the information related to the selected item opens.", + "p0044": "For example, when preventative maintenance item 52 is tapped and selected, the search method selection window 54 related to preventative maintenance of the machine tool 100 opens, as shown in FIG. 7 . The search method selection window 54 includes a time search button 56 for searching by time, an appearance search button 58 for searching based on the appearance of the machine tool 100 , a unit search button 60 for searching based on the units constituting the machine tool 100 , an index button 62 for searching by keyword, and a movie search button 64 for searching video files explaining inspection methods and methods for replacing parts by means of a video. A similar search method selection window is opened when another item is selected.", + "p0045": "When the operator taps and selects, for example, the unit search button 60 in the search method selection window 54 , the unit search window 70 of FIG. 8 opens. The unit search window 70 is constituted by a list of the plurality of units constituting the machine tool 100 , such as, for example, a spindle head, an automatic grease supply device, a pallet seating confirmation device, an ATC arm unit, a ring-type tool magazine, a spindle cooling oil temperature controller, an oil/air supply device, a hydraulic device, a pneumatic device, a chip processor/conveyor, a cutting fluid supply device/tank, and a splashguard.", + "p0046": "When the operator taps and selects, for example, the item related to the spindle device from the units displayed in the unit search window 70 , as shown in FIG. 9 , the item related to the spindle device opens, and a list window 72 of inspection items related to the spindle 112 of the machine tool 100 is displayed. In FIG. 9 , the inspection item list of the spindle device includes lubrication of a spindle collet, inspection and cleaning of a tool shank and inspection of a tool pot, and cleaning of a spindle taper part.", + "p0047": "As described regarding the alarm window 20 and the periodic inspection window 30 , when the operator taps and selects one item or a plurality of items from the list window 72 of inspection items, and taps the selection button 12 , information related to the selected item is displayed on the touch panel 122 . When a plurality of items are selected, each of the plurality of items can be displayed in an individual tab. The information displayed on the touch panel 122 includes the name of the part in which the malfunction has occurred, the method for accessing the part in which the malfunction has occurred, the name, serial number, and replacement method of a part to be replaced, a method for accessing the part for which inspection should be performed, inspection methods such as visual inspection or confirmation of gauges, and the name, serial number, and replacement method of a part to be replaced.", + "p0048": "Furthermore, when the two-dimensional code generation button 14 is tapped after item selection, a two-dimensional code 40 which is similar to the two-dimensional code 40 described above is displayed on the touch panel 122 . By reading this two-dimensional code with a smartphone or tablet, information can be displayed on the smartphone or tablet via the Internet 160 .", + "p0049": "The list window of inspection items is not limited to a list window related to a single unit, as shown in FIG. 9 . The list windows 72 , 74 related to a plurality of units, as shown in FIG. 10 , or alternatively, inspection item list windows related to all of the units displayed in the unit search window 70 may be opened.", + "p0050": "Further, in the embodiment described above, the operator can display the search window 50 on the touch panel 122 , which enables the user to search and refer to the electronic manual as necessary. The search window 50 provides a plurality of search methods to the operator, and thus, in the present invention, the search window 50 constitutes an operator-operated input part.", + "p0051": "REFERENCE SIGNS LIST", + "p0052": "- 10 Main Window - 12 Selection Button - 14 Two-Dimensional Code Generation Button - 16 Search Button - 20 Alarm Window - 30 Periodic Inspection Window - 40 Two-Dimensional Code - 50 Search Window - 52 Preventative Maintenance Item - 54 Search Method Selection Window - 56 Time Search Button - 58 Appearance Search Button - 60 Unit Search Button - 62 Index Button - 64 Movie Search Button - 70 Unit Search Window - 100 Machine Tool - 122 Operation Panel - 124 Touch Panel - 150 Controller", + "features": { + "F1": "An electronic manual for a machine tool that displays information related to operation, maintenance, or inspection necessary for an operator.", + "F2": "The electronic manual includes a display and a controller for the display.", + "F3": "The controller is configured to count elapsed time since prior inspection for one or more inspection items.", + "F4": "The controller is configured to control the display to show a first window with a list of units constituting the machine tool, allowing an operator to select one of the units.", + "F5": "Upon selection of a unit, the display shows a second window with items related to operation, maintenance, or inspection of the selected unit, allowing the operator to search the electronic manual.", + "F6": "A third window is displayed without operator prompt when the counted elapsed time reaches a predetermined time for each inspection item, allowing selection and input of inspection items.", + "F7": "The electronic manual can be stored in a controller of the machine tool, an Internet-connected server, or a mobile terminal.", + "F8": "A two-dimensional code related to operation, maintenance, or inspection information is displayed on the display, enabling connection to a server via the Internet when read by a mobile terminal.", + "F9": "Information related to operation, maintenance, or inspection of the machine tool is also displayed on the mobile terminal when connected to the server.", + "F10": "A machine tool includes a table for the workpiece, a tool for machining, a display, and a controller for the machine tool.", + "F11": "The controller of the machine tool is configured to count elapsed time since prior inspection for each inspection item.", + "F12": "The controller of the machine tool is configured to display a first window with a list of units, a selectable second window for further details, and an automatic third window based on elapsed time for inspection items." + } + } + }, + { + "FAN": "85176564", + "CPC": [ + "G02B-027/00/93", + "G02B-027/01/72", + "G02B-2027/0141", + "G06F-003/01", + "G06F-003/01/2", + "G06F-003/01/3", + "G06F-003/0346", + "G06F-003/0481", + "G06F-003/0484", + "G06F-016/00" + ], + "Content": { + "title": "Information processing device, information processing method, and program for determining a user gaze", + "pa01": "There is provided an information processing device, an information processing method, and a program are provided that enable displaying predetermined contents at the user-intended timing. The information processing device includes a viewpoint information obtaining unit that obtains viewpoint information related to the viewpoint of a user; a position-related information obtaining unit that obtains position-related information related to the position of the user and the position of a predetermined object; a head rotation information obtaining unit that obtains head rotation information related to the angular velocity accompanying the rotation of the head region of the user and related to the changes in the angular velocity; and a determining unit that, based on a gazing score which is calculated for each object according to the position-related information and the head rotation information and which indicates the extent of gazing of the user with respect to the object, determines whether or not the user is gazing at the object.", + "c-en-0001": "An information processing device comprising: circuitry configured to obtain viewpoint information related to a viewpoint of a user, obtain position-related information related to position of the user and position of predetermined object, obtain head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity, and determine, based on a gazing score which is calculated for the object according to the position-related information, the head rotation information, and object information related to the object, and which indicates extent of gazing of the user with respect to the object, whether or not the user is gazing at the object, wherein the object information related to the object on which the gazing score is based includes priority information related to the object, the priority information being based on a registered schedule of the user, and wherein the registered schedule of the user is stored as a result of performing machine learning of behavior of the user.", + "c-en-0002": "The information processing device according to claim 1 , wherein, when the gazing score for the object is equal to or greater than a predetermined threshold value, the circuitry determines that gazing state is attained in which the user is gazing at the object.", + "c-en-0003": "The information processing device according to claim 2 , wherein the predetermined threshold value for the object is determined according to a relation between the object and the registered schedule of the user.", + "c-en-0004": "The information processing device according to claim 1 , wherein the circuitry is further configured to calculate a gazing score based on the viewpoint information, the position-related information, and the head rotation information.", + "c-en-0005": "The information processing device according to claim 4 , wherein the circuitry calculates the gazing score based on a gazing score calculated at previous calculation timing, and based on a gazing score addition amount corresponding to the viewpoint information, the position-related information, and the head rotation information.", + "c-en-0006": "The information processing device according to claim 5 , wherein the circuitry calculates the gazing score according to distance between the viewpoint of the user and predetermined position of the object, the distance being calculated from the viewpoint information and the position-related information.", + "c-en-0007": "The information processing device according to claim 6 , wherein the circuitry calculates the gazing score according to distance between the viewpoint of the user and a center of the object, the distance being calculated from the viewpoint information and the position-related information.", + "c-en-0008": "The information processing device according to claim 6 , wherein the circuitry calculates the gazing score according to distance between the viewpoint of the user, which is on outside of the object, and edge of the object, the distance being calculated from the viewpoint information and the position-related information.", + "c-en-0009": "The information processing device according to claim 5 , wherein the circuitry calculates the gazing score based on the object information related to the object which is set in advance.", + "c-en-0010": "The information processing device according to claim 5 , wherein the circuitry calculates the gazing score based on user information which characterizes preferences of the user and which is set in advance.", + "c-en-0011": "The information processing device according to claim 5 , wherein the circuitry calculates the gazing score based on gazing score history information related to history of the gazing score.", + "c-en-0012": "The information processing device according to claim 5 , wherein the position-related information contains distance information related to distance between the user and the object, and the circuitry calculates the gazing score according to the distance information.", + "c-en-0013": "The information processing device according to claim 4 , wherein the position-related information contains relative velocity information related to relative velocity of the user with respect to the object, and the circuitry calculates the gazing score according to the relative velocity information.", + "c-en-0014": "The information processing device according to claim 4 , wherein the circuitry is further configured to control at least either a manner of displaying the object according to the gazing score of the object or display content of the object related to the object as provided according to the gazing score, wherein information related to the object for which the gazing score is calculated by the circuitry is stored as object history information, wherein the circuitry refers to the stored object history information and outputs object information containing an image of the object matching with predetermined condition.", + "c-en-0015": "The information processing device according to claim 1 , wherein the circuitry is further configured to control at least either a manner of displaying the object according to the gazing score of the object or display content of the object related to the object as provided according to the gazing score.", + "c-en-0016": "The information processing device according to claim 15 , wherein when the gazing score for the object is equal to or greater than predetermined threshold value, the circuitry determines that gazing state is attained in which the user is gazing at the object, and regarding the object under the gazing state, when the gazing score of the object is equal to or greater than predetermined threshold value, the circuitry displays at least a portion of the object information related to the object as provided according to the gazing score.", + "c-en-0017": "The information processing device according to claim 15 , wherein, when the gazing score is equal to or greater than predetermined threshold value, the circuitry displays at least a portion of the object information, which is related to the object and which is provided according to the gazing score, in a superimposed manner within field of view of the user.", + "c-en-0018": "The information processing device according to claim 15 , wherein the circuitry displays, in an enlarged manner, the object for which the gazing score is equal to or greater than threshold value.", + "c-en-0019": "The information processing device according to claim 15 , wherein a plurality of sets of information regarding each object of a plurality of objects is stored as the object information, and the circuitry displays at least a portion of the object information according to a predetermined condition determined from among the object information.", + "c-en-0020": "An information processing method comprising: obtaining viewpoint information related to a viewpoint of a user; obtaining position-related information related to position of the user and position of predetermined object; obtaining head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity; and determining, based on a gazing score which is calculated for the object according to the position-related information, the head rotation information, and object information related to the object, and which indicates extent of gazing of the user with respect to the object, whether or not the user is gazing at the object, wherein the object information related to the object on which the gazing score is based includes priority information related to the object, the priority information being based on a registered schedule of the user, and wherein the registered schedule of the user is stored as a result of performing machine learning of behavior of the user.", + "c-en-0021": "A non-transitory computer-readable storage medium having embodied thereon a program, which when executed by a computer causes the computer to execute a method, the method comprising: obtaining viewpoint information related to a viewpoint of a user; obtaining position-related information related to position of the user and position of predetermined object; obtaining head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity; and determining, based on a gazing score which is calculated for the object according to the position-related information, the head rotation information, and object information related to the object, and which indicates extent of gazing of the user with respect to the object, whether or not the user is gazing at the object, wherein the object information related to the object on which the gazing score is based includes priority information related to the object, the priority information being based on a registered schedule of the user, and wherein the registered schedule of the user is stored as a result of performing machine learning of behavior of the user.", + "p0001": "CROSS REFERENCE TO PRIOR APPLICATION", + "p0002": "This application is a National Stage Patent Application of PCT International Patent Application No. PCT/JP2018/041436 under 35 U.S.C. \u00a7 371, which claims priority to Japanese Patent Application No. 2018-000262 , which are all hereby incorporated by reference in their entirety.", + "p0003": "FIELD", + "p0004": "The application concerned is related to an information processing device, an information processing method, and a program.", + "p0005": "BACKGROUND", + "p0006": "In the case of detecting the user viewpoint with respect to a target group of objects for selection, such as menu items in the virtual space or objects in the real space, and then selecting an object; generally a method is used in which the holding time of the point of gaze toward each object is used to determine whether or not the gazing state is attained for that object. In the method for selecting an object by using the holding time of the point of gaze, smaller the determination threshold value for the holding time of the point of gaze, the higher is the possibility that an object different than the user-intended object gets selected.", + "p0007": "In that regard, in Patent Literature 1 mentioned below, an information processing device is disclosed that changes the display form of the UI , which is displayed in a display unit, according to the detection accuracy of the gazing position calculated using face orientation information of the user, distance information, gazing period information, and resolution information.", + "p0008": "CITATION LIST", + "p0009": "Patent Literature", + "p0010": "Patent Literature 1: JP 2016-151798 A", + "p0011": "SUMMARY", + "p0012": "Technical Problem", + "p0013": "The information processing device disclosed in Patent Literature 1 changes the display form according to the detection accuracy. However, if sufficient detection accuracy is not achieved, only a few icons are displayed near the gazing position of the user. Hence, there is still room for improvement m displaying predetermined contents at the user-intended timing.", + "p0014": "In that regard, in the application concerned, in view of the issues mentioned above, an information processing device, an information processing method, and a program are provided that enable displaying predetermined contents at the user-intended timing.", + "p0015": "Solution to Problem", + "p0016": "According to the present disclosure, an information processing device is provided that includes: a viewpoint information obtaining unit that obtains viewpoint information related to viewpoint of user; a position-related information obtaining unit that obtains position-related information related to position of the user and position of predetermined object; a head rotation information obtaining unit that obtains head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity; and a determining unit that, based on a gazing score which is calculated for each object according to the position-related information and the head rotation information and which indicates extent of gazing of the user with respect to the object, determines whether or not the user is gazing at the object.", + "p0017": "Moreover, according to the present disclosure, an information processing method is provided that includes: obtaining viewpoint information related to viewpoint of user; obtaining position-related information related to position of the user and position of predetermined object; obtaining head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity; and determining, based on a gazing score which is calculated for each object according to the position-related information and the head rotation information and which indicates extent of gazing of the user with respect to the object, whether or not the user is gazing at the object.", + "p0018": "Moreover, according to the present disclosure, a program is provided that causes a computer to function as: a viewpoint information obtaining unit that obtains viewpoint information related to viewpoint of user; a position-related information obtaining unit that obtains position-related information related to position of the user and position of predetermined object; a head rotation information obtaining unit that obtains head rotation information related to angular velocity accompanying rotation of head region of the user and related to changes in the angular velocity; and a determining unit that, based on a gazing score which is calculated for each object according to the position-related information and the head rotation information and which indicates extent of gazing of the user with respect to the object, determines whether or not the user is gazing at the object.", + "p0019": "According to the application concerned, a gazing score for each object is determined according to viewpoint information of the user, position-related information regarding the position of the user and the position of the object, and head rotation information regarding the rotation of the head region of the user.", + "p0020": "Advantageous Effects of Invention", + "p0021": "As described above, according to the application concerned, it becomes possible to display predetermined contents at the user-intended timing.", + "p0022": "The abovementioned effect is not necessarily limited in scope and, in place of or in addition to the abovementioned effect, any other effect indicated in the present written description or any other effect that may occur from the present written description can also be achieved.", + "p0023": "BRIEF DESCRIPTION OF DRAWINGS", + "p0024": "FIG. 1 is a block diagram illustrating a configuration example of an information processing system 1 according to an embodiment of the application concerned.", + "p0025": "FIG. 2 is an explanatory diagram for explaining the overview of head rotation information according to the concerned embodiment.", + "p0026": "FIG. 3 is an explanatory diagram for explaining the overview of head rotation information according to the concerned embodiment.", + "p0027": "FIG. 4 is an explanatory diagram for explaining the overview of head rotation information according to the concerned embodiment.", + "p0028": "FIG. 5 is an explanatory diagram for explaining the overview of head rotation information according to the concerned embodiment.", + "p0029": "FIG. 6A is an explanatory diagram for explaining an example of the display form of an object ob according to the concerned embodiment.", + "p0030": "FIG. 6B is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0031": "FIG. 6C is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0032": "FIG. 6D is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0033": "FIG. 6E is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0034": "FIG. 7 is an explanatory diagram for explaining the display method for displaying object information according to the concerned embodiment.", + "p0035": "FIG. 8 is an explanatory diagram for explaining the display method for displaying object information according to the concerned embodiment.", + "p0036": "FIG. 9 is an explanatory diagram for explaining the display method for displaying object information according to the concerned embodiment.", + "p0037": "FIG. 10A is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0038": "FIG. 10B is an explanatory diagram for explaining an example of the display form of the object ob according to the concerned embodiment.", + "p0039": "FIG. 11 is a flowchart for explaining an example of the flow of operations performed according to the concerned embodiment.", + "p0040": "FIG. 12 is an explanatory diagram for explaining an example of the operations according to the concerned embodiment.", + "p0041": "FIG. 13 is an explanatory diagram for explaining an example of the operations according to the concerned embodiment.", + "p0042": "FIG. is an explanatory diagram for explaining an example of user information according to the concerned embodiment.", + "p0043": "FIG. 15 is an explanatory diagram for explaining an example of the object information according to the concerned embodiment.", + "p0044": "FIG. 16 is an explanatory diagram for explaining an example of the operations according to the concerned embodiment.", + "p0045": "FIG. 17 is an explanatory diagram for explaining an example of gazing score history information according to the concerned embodiment.", + "p0046": "FIG. 18 is a block diagram illustrating a configuration of an information processing system 5 according to another embodiment of the application concerned.", + "p0047": "FIG. 19 is a block diagram illustrating an exemplary configuration of a display control device 550 according to the other embodiment.", + "p0048": "FIG. 20 is a flowchart for explaining an example of the flow of operations performed according to the other embodiment.", + "p0049": "FIG. 21 is an explanatory diagram for explaining an example of the operations according to the other embodiment.", + "p0050": "FIG. 22 is a block diagram illustrating a hardware configuration of the information processing device 10 according to the embodiments of the application concerned.", + "p0051": "DESCRIPTION OF EMBODIMENTS", + "p0052": "Preferred embodiments of the application concerned are described below in detail with reference to the accompanying drawings. In the present written description and the drawings, the constituent elements having practically identical functional configuration are referred to by the same reference numerals, and the explanation is not given repeatedly.", + "p0053": "Moreover, in the present written description and the drawings, a plurality of constituent elements having practically identical functional configuration are sometimes distinguished by attaching different alphabets after the same reference numerals. However, when a plurality constituent elements having practically identical functional configuration need not be particularly distinguished from each other, only the same reference numerals are used.", + "p0054": "The explanation is given in the following sequence.", + "p0055": "<<<First embodiment>>>", + "p0056": "<<Configuration>>", + "p0057": "<Overall configuration>", + "p0058": "<Configuration of information processing device 10 >", + "p0059": "<<Operations>>", + "p0060": "<<Operation example 1>>", + "p0061": "<<Operation example 2>>", + "p0062": "<<Operation example 3>>", + "p0063": "<<Operation example 4>>", + "p0064": "<<Operation example 5>>", + "p0065": "<<Operation example 6>>", + "p0066": "<<Operation example 7>>", + "p0067": "<<<Second embodiment>>>", + "p0068": "<<Configuration>>", + "p0069": "<<Operation>>", + "p0070": "<<<Hardware configuration>>>", + "p0071": "<<<Summary>>>", + "p0072": "First Embodiment", + "p0073": "<<Configuration>>", + "p0074": "Firstly, explained below with reference to FIG. 1 , FIG. 2 , and FIG. 3 is an overall configuration of an information processing system 1 according to a first embodiment of the application concerned. FIG. 1 is a block diagram illustrating a configuration example of the information processing system 1 according to the first embodiment of the application concerned. FIG. 2 is an explanatory diagram for explaining the overview of head rotation information according to the first embodiment. FIG. 3 is an explanatory diagram for explaining the overview of head rotation information according to the first embodiment.", + "p0075": "<Overall Configuration>", + "p0076": "As illustrated in FIG. 1 , the information processing system 1 according to the first embodiment includes at least an information processing device 10 . The information processing device 10 is capable of performing communication with one or more types of servers 20 , as may be necessary, via a known network 30 such as the Internet.", + "p0077": "The information processing device 10 according to the first embodiment can be a wearable device for a user U, such as an eyeglasses-type display or a head mount display ; or can be an information processing terminal such as a smartphone. The information processing device 10 obtains viewpoint information related to a viewpoint P of the user U, and obtains position-related information about the position of the user U and the position of an object ob. Moreover, based on the obtained viewpoint information and the obtained position-related information, the information processing device 10 obtains head rotation information that is related to the angular velocity and the changes in the angular velocity accompanying the rotation of the head region of the user U. The information processing device has the function of calculating a gazing score S, which indicates the extent of gazing at the object ob by the user U, according to the viewpoint information, the position-related information, and the head rotation information; and then determining whether or not the user U is gazing at the object ob.", + "p0078": "In each server 20 , object information is stored that represents a variety of information related to various objects ob present in the environment in which the user U is also present. Moreover, the server 20 can also manage user information of the user U of the information processing device 10 . The information processing device 10 can perform mutual communication with the appropriate server 20 as may be necessary, and can obtain information from that server 20 .", + "p0079": "The network 30 is implemented using a known information communication technology such as the Internet, and connects the information processing device 10 to the servers 20 .", + "p0080": "The object information contains at least either information related to the actual object ob or information with respect to the object ob that is set for use. The information related to the actual object ob can also contain, for example, the name of the object ob. Moreover, for example, when the object ob is a product, the information set with respect to the object ob can contain information such as the quantity of inventory, the sale priority, and the sale period.", + "p0081": "Furthermore, the object information can be registered in advance by the administrator of the object ob. For example, when the object ob is an object, the object information can be registered in advance by the manufacturer or the distribution firm of the object ob.", + "p0082": "The user information represents information characterizing the individual user U, such as the external appearance or the preferences of the user U. Such information can contain, for example, information such as the gender, the height, and the hobbies of the user U as well as information such as the schedule of the user U. The user information can be registered by the user U of the information processing device 10 , or can by automatically stored as a result of performing machine learning of the behavior of the user U by the server 20 .", + "p0083": "<Configuration of Information Processing Device 10 >", + "p0084": "Given below is the detailed explanation of a configuration of the information processing device 10 according to the first embodiment. For example, the information processing device 10 according to the first embodiment includes a viewpoint information obtaining unit 101 , a position-related information obtaining unit 103 , a head rotation information obtaining unit 105 , a gazing score calculating unit 107 , a determining unit 109 , a display control unit 111 , a communication control unit 113 , and a memory unit 115 .", + "p0085": "The viewpoint information obtaining unit 101 obtains viewpoint information related to the viewpoint P of the user U of the information processing device 10 based on image information obtained from an imaging device such as a visible light camera or an infrared camera that is installed in the information processing device 10 .", + "p0086": "In the information processing device 10 , a three-dimensional rectangular coordinate system is set in advance as the coordinate system for the information processing device 10 . For example, the viewpoint information obtaining unit 101 can obtain the positions of the eyes and the eye gaze direction of the user U in the device coordinate system using a known viewpoint detection technology, and can treat the obtained information as the viewpoint information by converting the device coordinate system into the global coordinate system representing the fixed coordinate system set in the space. The obtained viewpoint information is sent to at least either the head rotation information obtaining unit 105 or the gazing score calculating unit 107 . Moreover, the obtained viewpoint information can be sent to the memory unit 115 as may be necessary, and can be stored as history information.", + "p0087": "The position-related information obtaining unit 103 obtains the position-related information that is related to the position of the user U. The position-related information contains a variety of information related to the position of the user U. Examples of such information include the distance between the user U and the object ob and the movement speed of the user U with respect to the object ob .", + "p0088": "The position-related information obtaining unit 103 is implemented using, for example, the camera installed in the information processing device 10 ; and the position and the orientation of the device coordinate system, when viewed from the global coordinate system set in the space in which the information processing device 10 is installed, can be obtained using a known technology such as the SLAM . The obtained position and the obtained orientation of the device coordinate system are used as the position and the orientation of the user U. Then, the obtained position-related information is sent to at least either the head rotation information obtaining unit 105 or the gazing score calculating unit 107 . Moreover, the obtained position-related information can be sent to the memory unit 115 as may be necessary, and can be stored as history information.", + "p0090": "The head rotation information obtaining unit 105 obtains head rotation information related to the angular velocity accompanying the rotation of the head region of the user U and the changes occurring in the angular velocity.", + "p0091": "Herein, for example, the head rotation information obtaining unit 105 can obtain the direction of the device coordinate system, as obtained by the position-related information obtaining unit 103 , as the direction of the head region. Moreover, the head rotation information obtaining unit 105 can obtain the rotation direction of the head region of the user U by detecting the temporal changes in the direction of the head region. More specifically, the head rotation information is obtained based on the straight line joining the head rotation center that can be treated as the position of the user U and an object center C, and based on a head direction ud of the user U obtained by the position-related information obtaining unit 103 .", + "p0092": "FIG. 2 is a diagram that schematically illustrates a case in which the user U attempts to gaze at a predetermined object ob.", + "p0093": "The head rotation center of the user U and the center of the object ob are obtained by the position-related information obtaining unit 103 using a known position obtaining technology. The rotation direction of the head region of the user U is obtained by the head rotation information obtaining unit 105 by detecting the temporal changes in the head direction ud of the user U with respect to the position of the user U, that is, with respect to the straight line joining the head rotation center and the object center C.", + "p0094": "Moreover, the head rotation information obtaining unit 105 detects an angle \u03b1 t1 at a timing t 1 and detects an angle \u03b1 t2 at a timing t 2 different than the timing t 1 ; and calculates an angular velocity \u03c9 by dividing the difference \u03b1 t2 \u2212\u03b1 t1 between the angles \u03b1 t2 and \u03b1 t2 by t 2 \u2212t 1 .", + "p0095": "The head rotation information obtained by the head rotation information obtaining unit 105 is sent to the gazing score calculating unit 107 . Moreover, the obtained head rotation information can be sent to the memory unit 115 as may be necessary, and can be stored as history information.", + "p0096": "Meanwhile, the position-related information obtaining unit 103 and the head rotation information obtaining unit 105 can be configured using a sensor such as an acceleration sensor or a gyro sensor meant for obtaining position information; and the position-related information and the head rotation information about the user U can be obtained using the output from the sensor.", + "p0098": "The gazing score calculating unit 107 calculates, for each object ob, the gazing score S, which indicates the extent of gazing at the object ob by the user U, according to at least the viewpoint information obtained by the viewpoint information obtaining unit 101 , the position-related information obtained by the position-related information obtaining unit 103 , and the head rotation information obtained by the head rotation information obtaining unit 105 . The calculated gazing score S is sent to at least either the determining unit 109 or the display control unit 111 .", + "p0099": "Given below is the explanation of an overview of the calculation method for calculating the gazing score. As described above, the gazing score S is calculated using the rotation direction of the head region of the user U and angular velocity \u03c9 accompanying that rotation.", + "p0100": "In addition to using the rotation direction of the head region of the user U and the angular velocity \u03c9 accompanying the rotation, the calculation of the gazing score S can be set also based on all types of information handled by the information processing device 10 according to the first embodiment. For example, the calculation score S can be calculated according to the following: the distance between the user U and the object ob representing one part of the position-related information; the distance d from the object center C to the viewpoint P as calculated based on the viewpoint information and the position-related information; and the user information containing information about the characteristics of the user U and information that characterizes the preferences of the user U.", + "p0101": "When the user U attempts to gaze at a predetermined object ob, he or she rotates the head region toward the central direction of that object ob with the aim of matching the direction of the head region with the direction toward the object ob. Subsequently, when the direction of the head region of the user U moves closer to the direction toward the object ob, the user U slows down the rotating speed of the head region. That is, when the user U attempts to gaze at a predetermined object ob, an angle \u03b1, which represents the difference between the straight line joining the object center C to the user U and the head direction ud as illustrated in FIG. 2 , goes on decreasing; and, as the angle \u03b1 1 becomes smaller, the user U reduces the angular velocity \u03c9 accompanying the rotation of the head region.", + "p0102": "When the user U does not gaze at the object ob, as illustrated in FIG. 3 , even if the head region is rotated toward the direction of the object ob, he or she rotates the head region without reducing the angular velocity \u03c9 in the vicinity of the portion in which the head direction ud of the user U matches with the object center C. Then, the user U keeps on rotating the head region in such a way that the angle \u03b1 increases.", + "p0103": "Moreover, from the state of gazing at a predetermined object ob, when the user changes the gazing target to another object ob, he or she rotates the head region so as to match the direction of the head direction with the direction of that other object ob. At that time, the head region of the user u rotates in such a way that the angular velocity \u03c9 increases.", + "p0104": "Thus, as a result of focusing on the rotation direction of the head region and the angular velocity \u03c9 with respect to the object ob, it becomes possible to accurately and promptly calculate the gazing score S in which the intention of the user U is reflected. That is, as a result of setting the calculation formula for calculating the gazing score S to reflect the rotation direction of the head region and the angular velocity \u03c9, the gazing score S can be calculated in an accurate and prompt manner.", + "p0105": "Moreover, the gazing score calculating unit 107 can calculate the gazing score S according to the position of the viewpoint of the user U.", + "p0106": "For example, in the case in which the viewpoint P of the user U is set on the object ob and in which the rotation direction of the head region of the user U with respect to the object ob is toward a decrease in the angle \u03b1 formed by the straight line joining the head rotation center and the object center C with the direction of the head region of the user U as obtained by the position-related information obtaining unit 103 ; the gazing score calculating unit 107 can increase the gazing score S.", + "p0107": "Alternatively, even when the viewpoint P of the user U is set on the object ob and when the rotation direction of the head region of the user U with respect to the object ob is toward an increase in the angle \u03b1, as long as the angular velocity \u03c9 accompanying the rotation of the head region of the user U is equal to or smaller than a threshold value \u03c9 th , the gazing score calculating unit 107 can increase the gazing score S.", + "p0108": "The gazing score calculation formula can be varied for each timing of calculating the gazing score S. While the user U is moving the viewpoint P and rotating the head region with the aim of gazing at an object ob; the viewpoint information, the position-related information, and the head rotation information changes according to the timing of calculating the gazing score S. Hence, for example, for each timing of calculating the gazing score S, the gazing score calculating unit 107 can vary the gazing score calculation formula according to the distance between the object on and the user U, the distance d from the object center C to the viewpoint P of the user U, and the angle \u03b1 formed by the straight line joining the head rotation center and the object center C with the direction in which the user U is facing.", + "p0109": "For example, when the user U attempts to gaze at a predetermined object ob, if the angle \u03b1 is small, a calculation formula can be used that increases the gazing score S as compared to the case in which the angle \u03b1 is large. Moreover, when the angle \u03b1 is small and the angular velocity \u03c9 too is low, a calculation formula can be used that further increases the gazing score S.", + "p0110": "Furthermore, for example, when the viewpoint P of the user U is not set on the object ob, a gazing score calculation formula can be used that reduces the gazing score S. As a result, regarding the objects ob that were already excluded from the gazing targets for the user U, it becomes possible to hold down the selection of the objects ob not intended by the user U.", + "p0111": "The gazing score calculating unit 107 can calculate a gazing score addition amount according to the viewpoint information, the position-related information, and the head rotation information that is obtained; add the gazing score addition amount to the gazing score S obtained immediately before; and again calculate the gazing score S.", + "p0112": "More particularly, the gazing score calculating unit 107 can calculate the gazing score addition amount according to the gazing score S calculated at the timing t 1 and according to the head rotation information obtained at the timing t 2 that is different than the timing t 1 .", + "p0113": "Moreover, the initial value of the gazing score S and the gazing score addition amount can vary for each timing of calculating the gazing score S. For example, the gazing score calculating unit 107 can vary the initial value of the gazing score S and the gazing score addition amount according to the distance between the object ob and the user U, the distance from the object center C to the viewpoint P of the user U, and the angle \u03b1 formed by the straight line joining the head rotation center and the object center C with the direction in which the user U is facing.", + "p0114": "More particularly, if S t represents the gazing score S at a timing t and if A represents the gazing score addition amount, then the gazing score S can be calculated according to an equation given below. Herein, f can be treated as an arbitrary function having the score S t as the variable. Although there is no restriction on the specific function form, it is possible to use a polynomial function such as a linear function or a quadratic function, or it is possible to use an exponent function or a logarithmic function.", + "p0115": "S t2 =f + A", + "p0116": "The gazing score addition amount A can be a fixed value or can be varied according to the head rotation information. In addition to varying it according to the head rotation information, the gazing score addition amount can be dynamically varied according to the user information such as the distance between the user U and the object ob representing one part of the position-related information, or the distance d from the object center C to the viewpoint P of the user U as calculated based on the viewpoint information and the position-related information, or the characteristics of the user U and information characterizing the preferences of the user U.", + "p0117": "For example, in the case of making the gazing score addition amount A inversely proportional to the angular velocity \u03c9 representing the head rotation information, the gazing score addition amount A can be obtained using an angular velocity \u03c9 t at the timing t and an adjustment coefficient c as given below in an equation. The adjustment coefficient c can be set as a value that varies according to the distance between the user U and the object ob, or the distance d from the object center C to the viewpoint P of the user U, or the characteristics of the user U and the user information.", + "p0118": "A=c/\u03c9 t", + "p0119": "Herein, \u03c9 t \u22600 holds true.", + "p0120": "Meanwhile, the calculated gazing score S can be sent to the memory unit 115 , and can be stored as history information.", + "p0122": "Based on the gazing score S calculated by the gazing score calculating unit 107 , the determining unit 109 determines, for each object ob, whether or not the user U is gazing at the object ob. More particularly, it is desirable that, when the gazing score S of an object ob is equal to or greater than a predetermined determination threshold value, the determining unit 109 determines that the gazing state is attained in which the user U is gazing at the object ob. The determination result is sent at least to the display control unit 111 , and can also be sent to the memory unit 115 as may be necessary and stored as history information.", + "p0123": "Explained below with reference to FIGS. 4 and 5 is the manner in which changes occur in the gazing score S calculated by the gazing score calculating unit 107 and the gazing determination performed by the determining unit 109 . FIG. 4 is a graph illustrating an example of the angular velocity \u03c9 of the user at each timing t. FIG. 5 is a graph illustrating the relationship between each timing t and the gazing score S calculated by the gazing score calculating unit 107 . Herein, areas R 1 to R 5 illustrated in FIG. 4 correspond to areas R 1 \u2032 to R 5 \u2032, respectively, illustrated in FIG. 5 . Moreover, S 1 th illustrated in FIG. 5 represents the determination threshold value set in advance. When the gazing score S becomes equal to or greater than the determination threshold value S 1 th , the determining unit 109 determines that the gazing state is attained in which the user U is gazing at the object ob.", + "p0124": "Explained below with reference to an example is the case of calculating the gazing score addition amount when the viewpoint P of the user U is on the target object ob. The gazing score S has the initial value of zero. When the viewpoint P of the user U is not on the target object ob for gazing , the gazing score addition amount is set to have a negative value; and when the gazing score S is equal to zero, the gazing score addition amount is set to zero. Moreover, the gazing determination method is explained under the assumption that the calculated gazing score addition amount is constant.", + "p0125": "As illustrated in FIG. 4 , the plane defined by the timing t and the angular velocity \u03c9 can be divided into areas R 1 to R 5 . The area R 1 is an area indicating the state in which angle \u03b1, which is formed by the straight line joining the head rotation center and the object rotation center C with the head direction ud of the user U as obtained by the position-related information obtaining unit 103 , rotates in the direction of decrease, and the angular velocity \u03c9 accompanying the rotation of the head is equal to or greater than the predetermined threshold value \u03c9 th ; but the viewpoint P of the user U is not on the gazing target object ob. The area R 2 is an area indicating the state in which the viewpoint P of the user U is on the target object ob for gazing, the angle \u03b1 rotates in the direction of decrease, and the angular velocity \u03c9 is equal to or greater than the predetermined threshold value \u03c9 th . The area R 3 is an area indicating the state in which the viewpoint P of the user U is on the target object ob for gazing, the angle \u03b1 rotates in the direction of decrease, and the angular velocity \u03c9 is smaller than the predetermined threshold value \u03c9 th . The area R 4 is an area indicating the state in which the viewpoint P of the user U is on the target object ob for gazing, the angle \u03b1 rotates in the direction of increase, and the angular velocity \u03c9 is smaller than the predetermined threshold value \u03c9 th . The area R 5 is an area indicating the state in which the viewpoint P of the user U is not on the target object ob for gazing, the angle \u03b1 rotates in the direction of increase, and the angular velocity \u03c9 is equal to or greater than the predetermined threshold value \u03c9 th .", + "p0126": "In the area R 1 , since the viewpoint P of the user U is not on the target object ob, as illustrated in FIG. 5 , in an area R 1 \u2032 corresponding to the area R 1 , the gazing score addition amount is equal to zero and the gazing score S is constant at zero.", + "p0127": "In the area R 2 , although the viewpoint P of the user U is on the target object ob for gazing and the angular velocity \u03c9 is equal to or greater than the predetermined threshold value \u03c9 th , the head region of the user U is rotating in the direction of decrease in the angle \u03b1. Hence, as illustrated in an area R 2 \u2032, the gazing score S increases. Herein, the extent of increase in the gazing score S is decided according to the function form of f in Equation .", + "p0128": "In the area R 3 , the viewpoint P of the user U is on the target object ob for gazing, the angular velocity \u03c9 is smaller than the predetermined threshold value \u03c9 th , and the head region of the user U is rotating in the direction of decrease in the angle \u03b1. Hence, the gazing score S increases.", + "p0129": "In the area R 4 , although the viewpoint P of the user U is on the target object ob for gazing and the head region of the user U is rotating in the direction of increase in the angle \u03b1, the angular velocity \u03c9 is smaller than the predetermined threshold value \u03c9 th . Hence, the gazing score S increases. At a timing t\u2032, the gazing score S becomes equal to or greater than the determination threshold value S 1 th , and thus the determining unit 109 determines that the gazing state is attained in which the user U is gazing at the object ob.", + "p0130": "Even after the gazing score S has reached the determination threshold value S 1 th , the score calculation is continued. Alternatively, after having reached the determination threshold value S 1 th , the gazing score S can be maintained at the threshold value.", + "p0131": "The area R 5 indicates the state in which the viewpoint P of the user U moves away from the object ob under the gazing state to another object ob. Since the viewpoint P of the user U has moved away from the object ob under the gazing state, the gazing score S decreases.", + "p0132": "When the viewpoint P of the user U moves away from the object ob having the gazing score S greater than the threshold value, the gazing score calculating unit 107 can calculate the gazing score S in such a way that there is a decrease in the gazing score S as illustrated in FIG. 5 . When the viewpoint P of the user U moves away from the object ob, the gazing score S decreases and, when it becomes smaller than the determination threshold value S 1 th , the gazing state can be terminated. The operations can be performed in such a way that the gazing score S becomes equal to zero immediately after the away movement of the viewpoint P. As a result of performing such operations, immediately after the target object ob for gazing for the user U has been changed from the object ob being gazed to another object ob, the gazing state can be terminated.", + "p0134": "The display control unit 111 controls, according to the gazing score S for the object ob, at least either the display method for displaying that object ob in the display screen or the display content regarding that object ob.", + "p0135": "For example, as illustrated in FIGS. 6A to 6E , the display control unit 111 controls the display method for the object ob according to the gazing score S. In FIG. 6A is illustrated an example in which the gazing score S for the object ob is displayed in the form of a meter display. The display control unit 111 can perform the display in such a way that the meter reading increases or decreases according to the increase or decrease in the gazing score S for the object ob.", + "p0136": "Meanwhile, in the case of displaying the selected object ob in an enclosed manner, the display control unit 111 can perform the display with an expression that makes visual recognition easier for the user U as the gazing score S goes on increasing. For example, as illustrated in FIGS. 6B and 6C , depending on the gazing score S, the display control unit 111 can highlight, in a phased manner, the object ob by changing the aspects such as the surrounding color, the transparency, and the selection expression of the object ob.", + "p0137": "Moreover, as illustrated in FIG. 6D , the display control unit 111 can change the display size of the target object ob according to the gazing score S.", + "p0138": "For example, an image of the object ob can be obtained using a camera that is installed in the information processing device 10 according to the first embodiment and that recognizes the external space; and the display control unit 111 can display that image in a superimposed manner on the object ob. Then, the display control unit 111 can change the size of the superimposed image of the object ob according to the gazing score S and provide it to the user U.", + "p0139": "Meanwhile, the number of objects ob that are displayed with a changed display size is not limited to one and it is also possible to display a plurality of objects ob by changing their sizes. When the gazing scores S for a plurality of objects ob are within the range of predetermined values, the display control unit 11 can display the objects ob in an enlarged manner.", + "p0140": "Furthermore, as illustrated in FIG. 6E , regarding an object ob that is only partially displayed, the display control unit 111 can display that object ob by moving it according to the gazing score S.", + "p0141": "For example, in an identical manner to the description given above, an image of the object ob is taken using the camera that is installed in the information processing device 10 according to the first embodiment and that recognizes the external space; and the display control unit 111 can provide the taken image, which is superimposed on the object ob, by moving it according to the gazing score S.", + "p0142": "The selection expressions explained above can be used in an individual manner, or can be used in combination. Moreover, in the application concerned, the expression methods that are not explained herein but that change according to the gazing score S are also assumed to be included.", + "p0143": "The selection expression that is displayed by the display control unit 111 can change on a continuous basis accompanying the increase or decrease of the gazing score S; or such a selection expression can be used for which one or more threshold values are set in advance and which changes in a phased manner. More particularly, when the gazing score S is equal to or greater than a predetermined threshold value, the display control unit 111 can display the object ob according to a predetermined selection expression. Moreover, regarding an object ob for which the gazing score S becomes equal to or smaller than the predetermined threshold value, the selection expression for that object ob can be reset.", + "p0144": "Meanwhile, as illustrated in FIGS. 7 to 9 , according to the gazing score S, the object information related to the object ob can be displayed. FIG. 7 is an explanatory diagram for explaining the display method for displaying the object information. FIGS. 8 and 9 are schematic diagrams illustrating examples of the image displayed in the object information.", + "p0145": "More particularly, the display control unit 111 can display, according to the gazing score S, the information set with respect to an object ob, such as the information containing the name and the category of the object ob. For example, when the object ob is a product, the display control unit 111 can display, according to the object ob, the information such as the quantity of inventory, the sale priority, and the sale period.", + "p0146": "As illustrated in FIG. 7 , display threshold values S 2 th are set with respect to the gazing score S for the purpose of displaying the object information and, when the gazing score S is equal to or greater than a display threshold value S 2 th , the display control unit 111 displays the object information at the corresponding information display level which enables superimposition of the object information within the eyesight of the user U. For example, when the gazing score S for the object ob becomes equal to or greater than a display threshold value S 2 th1 , as illustrated in the top left side in FIG. 8 , first-type object information belonging to an information display level 1 is displayed to the user U. The displayed first-type object information can be, for example, the fundamental information related to that object ob. More particularly, when the concerned object ob is a product such as a bag displayed in the shelf of a shop, the display control unit 111 can display the information such as the brand name and the price of that product.", + "p0147": "As illustrated in FIG. 7 , when the gazing score S for the object ob becomes equal to or greater than a display threshold value S 2 th2 , as illustrated in the bottom right side in FIG. 9 , second-type object information belonging to a display level 2 is displayed to the user U. As compared to the first-type object information, the second-type object information can be set to be more detailed information. For example, when the object ob is a product, the second-type object information can be set as the information related to objects ob that have the same category as the concerned object ob and that are already in possession of the user U. When the object ob is a product, the display control unit 111 can display the information about the attributes of the user U, such as the gender, the height, and the age of the user U who is recommended to use that product; can display the information about the situation in which the use of the product is recommended; and can display the information such as similar possessions of the user U to the concerned product.", + "p0148": "The display control unit 111 can remove the object information displayed immediately after the away movement of the viewpoint P of the user U from the concerned object ob. Moreover, also when the gazing score S becomes smaller than the predetermined display threshold value S 2 th , the display control unit 111 can remove the corresponding displayed object information.", + "p0149": "Furthermore, the display control unit 111 can vary the method of presenting the object information according to the gazing score S. FIGS. 10A and 10B are explanatory diagrams for explaining the method of presenting the object information according to the gazing score S.", + "p0150": "For example, when the viewpoint P of the user U is set on the object ob, as illustrated in FIG. 10A , a leader line is displayed to be emerging from the object ob. As the gazing score S becomes higher, the leader line becomes elongated and, when the information display level is reached, the object information can be displayed. Moreover, for example, as illustrated in FIG. 10B , a word balloon coming from the object ob can be displayed; the degree of transparency or the extent of blurring of the word balloon can be varied according to the gazing score S; and when the information display level is reached, predetermined object information can be displayed.", + "p0152": "The communication control unit 113 has a communication control function for obtaining, via the network 30 , the object information stored in the server 20 and the user information related to the attributes of the user U; and sending the object information and the user information to the display control unit.", + "p0154": "The memory unit 115 is an example of a memory device installed in the information processing device 10 according to the first embodiment. The memory unit 115 is used to register various programs and databases to be used by the information processing device 10 in performing various operations as described above. Moreover, the memory unit 115 can also be used to record, as history information, a variety of information obtained by the viewpoint information obtaining unit 101 and the position-related information obtaining unit 103 . Furthermore, the memory unit 115 can also be used to record various parameters and intermediate operation results that need to be stored during the operations performed by the gazing score calculating unit 107 and the determining unit 109 . Moreover, not limiting the information to be stored to the information handled during the operations performed by the gazing score calculating unit 107 and the determining unit 109 , the memory unit 115 can also be used to record various parameters and intermediate operation results that need to be stored during any operations performed in the information processing device 10 according to the first embodiment. The memory unit 115 is freely readable/writable for the viewpoint information obtaining unit 101 , the position-related information obtaining unit 103 , the head rotation information obtaining unit 105 , the gazing score calculating unit 107 , and the determining unit 109 .", + "p0155": "Till now, the detailed explanation was given about an exemplary configuration of the information processing device 10 . The abovementioned constituent elements can be configured using general-purpose members or circuits, or can be configured using hardware specialized to implement the functions of the constituent elements. Moreover, the functions of the constituent elements can be entirely implemented by a CPU. Thus, according to the technical level at the time of implementing the first embodiment, the configuration can be appropriately modified.", + "p0156": "Given below is the detailed explanation of an example of the operations performed in the information processing device 10 .", + "p0157": "<<Operations>>", + "features": { + "F1": "An information processing device comprises circuitry configured to obtain viewpoint information related to a viewpoint of a user.", + "F2": "The circuitry is configured to obtain position-related information related to the position of the user and the position of a predetermined object.", + "F3": "The circuitry is configured to obtain head rotation information related to angular velocity accompanying rotation of the head region of the user and related to changes in the angular velocity.", + "F4": "The circuitry is configured to determine, based on a gazing score calculated for the object using position-related information, head rotation information, and object information, whether or not the user is gazing at the object.", + "F5": "The object information includes priority information related to the object, based on a registered schedule of the user stored as a result of machine learning of the user's behavior.", + "F6": "When the gazing score for the object is equal to or greater than a predetermined threshold value, the circuitry determines that the user is gazing at the object.", + "F7": "The predetermined threshold value is determined according to a relation between the object and the registered schedule of the user.", + "F8": "The circuitry is configured to calculate a gazing score based on the viewpoint information, position-related information, and head rotation information.", + "F9": "The gazing score is calculated based on a previous gazing score and a gazing score addition amount corresponding to the viewpoint information, position-related information, and head rotation information.", + "F10": "The circuitry calculates the gazing score according to the distance between the viewpoint of the user and a predetermined position of the object.", + "F11": "The circuitry calculates the gazing score according to the distance between the viewpoint of the user and the center of the object.", + "F12": "The circuitry calculates the gazing score according to the distance between the viewpoint of the user and the edge of the object.", + "F13": "The circuitry calculates the gazing score based on object information set in advance.", + "F14": "The circuitry calculates the gazing score based on user information characterizing the user's preferences set in advance.", + "F15": "The circuitry calculates the gazing score based on gazing score history information.", + "F16": "The position-related information contains distance information related to the distance between the user and the object, and the circuitry calculates the gazing score according to the distance information.", + "F17": "The position-related information contains relative velocity information of the user with respect to the object, and the circuitry calculates the gazing score according to the relative velocity information.", + "F18": "The circuitry is configured to control the manner of displaying the object or display content according to the gazing score.", + "F19": "Information related to the object is stored as object history information, and the circuitry uses this information to output object information containing an image matching a predetermined condition.", + "F20": "When the gazing score for the object reaches or exceeds a threshold, the circuitry displays at least a portion of the object information in a superimposed manner within the user's field of view.", + "F21": "The circuitry displays the object for which the gazing score is equal to or greater than the threshold value in an enlarged manner.", + "F22": "A plurality of sets of information regarding each object is stored, and the circuitry displays at least a portion of the object information according to a predetermined condition.", + "F23": "An information processing method involves obtaining viewpoint, position-related, and head rotation information, and determining if the user is gazing at the object based on a gazing score.", + "F24": "A non-transitory computer-readable storage medium embodies a program that, when executed, causes a computer to obtain viewpoint, position-related, and head rotation information, and determine gazing based on a gazing score." + } + } + }, + { + "FAN": "74598812", + "CPC": [ + "G06F-016/00", + "G06F-016/735", + "G06F-016/7834", + "G06F-016/7837", + "G06F-017/30/787", + "G06F-017/30/828", + "G06F-018/22", + "G06K-009/00/744", + "G06K-009/00/758", + "G06K-009/46/04", + "G06K-009/46/47", + "G06K-009/62/02", + "G06K-009/62/15", + "G06K-009/62/32", + "G06V-010/44", + "G06V-010/462", + "G06V-010/761", + "G06V-020/46", + "G06V-020/48", + "G10L-025/51", + "G10L-025/54" + ], + "Content": { + "title": "Video detection method, server and storage medium", + "pa01": "Embodiments of this application disclose a video detection method and a server. The method includes: obtaining first video data to be detected; decoding the first video data, to obtain audio data of the first video data; analyzing and identifying the audio data, to obtain audio fingerprint data corresponding to the audio data; querying an audio fingerprint library; obtaining a video label and a time parameter corresponding to the audio data when the audio fingerprint library comprises the audio fingerprint data; querying a video copyright library, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracting a second picture that is in the first video data and that satisfies the time parameter; separately extracting a first feature parameter of the first picture and a second feature parameter of the second picture; and comparing the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "c-en-0001": "A video detection method, comprising: obtaining, by a server, first video data to be detected, and decoding, by the server, the first video data, to obtain audio data of the first video data; analyzing and identifying, by the server, the audio data, to obtain audio fingerprint data corresponding to the audio data; extracting, by the server, a picture corresponding to a predefined scene switching in second video data having copyright, further including: performing, by the server, scene identification on the second video data, and identifying and filtering a first picture collection representing the predefined scene switching in the second video data, to obtain a second picture collection; analyzing and identifying, by the server, a picture in the second picture collection, to obtain edge feature information of the picture in the second picture collection; and extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold; and storing the extracted picture, a time parameter corresponding to the extracted picture, and a video label corresponding to the extracted picture in a video copyright library; querying, by the server based on the audio fingerprint data, an audio fingerprint library, wherein the audio fingerprint library stores audio fingerprint data of copyrighted video data and video labels and time parameters corresponding to the audio fingerprint data; and in accordance with a determination that the audio fingerprint data exists in the audio fingerprint library: obtaining, by the server, a video label and a time parameter corresponding to the audio fingerprint data from the audio fingerprint library; querying, by the server, the video copyright library based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracting a second picture that is in the first video data and that satisfies the time parameter; separately extracting, by the server, a first feature parameter of the first picture and a second feature parameter of the second picture; and comparing, by the server, the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "c-en-0002": "The method according to claim 1 wherein the extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold comprises: generating, by the server, the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extracting, by the server, a picture in the third picture collection at a preset time interval.", + "c-en-0003": "The method according to claim 1 , wherein the analyzing and identifying, by the server, the audio data, to obtain audio fingerprint data corresponding to the audio data comprises: extracting, by the server, a feature parameter of the audio data, and obtaining, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "c-en-0004": "The method according to claim 1 , wherein the separately extracting, by the server, a first feature parameter of the first picture and a second feature parameter of the second picture comprises: separately extracting, by the server, the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a scale-invariant feature transform SIFT manner and a histogram of oriented gradient HOG manner.", + "c-en-0005": "The method according to claim 1 , wherein the comparing, by the server, the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library comprises: calculating, by the server, similarity between the first feature parameter and the second feature parameter; and determining that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "c-en-0006": "The method according to claim 1 , wherein the audio fingerprint data is quantized data representing a feature parameter of the audio data.", + "c-en-0007": "A server, comprising at least one processor and a memory storing a processor-executable instruction, the instruction, when executed by the at least one processor, causing the server to perform the following operations: obtaining first video data to be detected; decoding the first video data, to obtain audio data of the first video data; analyzing and identifying the audio data, to obtain audio fingerprint data corresponding to the audio data; extracting, by the server, a picture corresponding to a predefined scene switching in second video data having copyright, further including: performing, by the server, scene identification on the second video data, and identifying and filtering a first picture collection representing the predefined scene switching in the second video data, to obtain a second picture collection; analyzing and identifying, by the server, a picture in the second picture collection, to obtain edge feature information of the picture in the second picture collection; and extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold; and storing the extracted picture, a time parameter corresponding to the extracted picture, and a video label corresponding to the extracted picture in a video copyright library; querying, based on the audio fingerprint data, an audio fingerprint library, wherein the audio fingerprint library stores audio fingerprint data of copyrighted video data and video labels and time parameters corresponding to the audio fingerprint data; and; in accordance with a determination that the audio fingerprint data exists in the audio fingerprint library: obtaining a video label and a time parameter corresponding to the audio fingerprint data from the audio fingerprint library; querying a video copyright library based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracting a second picture that is in the first video data and that satisfies the time parameter; separately extracting a first feature parameter of the first picture and a second feature parameter of the second picture; and comparing the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "c-en-0008": "The server according to claim 7 , wherein the server is further configured to perform the following operations: generating the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extracting, by the server, a picture in the third picture collection at a preset time interval.", + "c-en-0009": "The server according to claim 7 , wherein the server is further configured to perform the following operations: extracting a feature parameter of the audio data, and obtaining, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "c-en-0010": "The server according to claim 7 , wherein the server is further configured to perform the following operations: separately extracting the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a scale-invariant feature transform SIFT manner and a histogram of oriented gradient HOG manner.", + "c-en-0011": "The server according to claim 7 , wherein the server is further configured to perform the following operations: calculating similarity between the first feature parameter and the second feature parameter; and determining that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "c-en-0012": "The server according to claim 7 , wherein the audio fingerprint data is quantized data representing a feature parameter of the audio data.", + "c-en-0013": "A non-transitory storage medium, configured to store one or more computer programs, the computer programs comprising an instruction that can be run by a possessor comprising one or more memories, the instruction, when executed by a computer, causing the computer to perform the following operations: obtaining first video data to be detected, and decoding the first video data, to obtain audio data of the first video data; analyzing and identifying the audio data, to obtain audio fingerprint data corresponding to the audio data; extracting, by the server, a picture corresponding to a predefined scene switching in second video data having copyright, further including: performing, by the server, scene identification on the second video data, and identifying and filtering a first picture collection representing the predefined scene switching in the second video data, to obtain a second picture collection; analyzing and identifying, by the server, a picture in the second picture collection, to obtain edge feature information of the picture in the second picture collection; and extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold; and storing the extracted picture, a time parameter corresponding to the extracted picture, and a video label corresponding to the extracted picture in a video copyright library; querying, based on the audio fingerprint data, an audio fingerprint library, wherein the audio fingerprint library stores audio fingerprint data of copyrighted video data and video labels and time parameters corresponding to the audio fingerprint data and; in accordance with a determination that the audio fingerprint data exists in the audio fingerprint library: obtaining a video label and a time parameter corresponding to the audio fingerprint data from the audio fingerprint library; querying a video copyright library based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracting a second picture that is in the first video data and that satisfies the time parameter; separately extracting a first feature parameter of the first picture and a second feature parameter of the second picture; and comparing the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "c-en-0014": "The method according to claim 13 , wherein the extracting the picture of which a quantity of the edge feature information reaches a preset threshold comprises: generating a picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extracting a picture in the third picture collection at a preset time interval.", + "p0001": "PRIORITY CLAIM AND RELATED APPLICATION", + "p0002": "This application is a continuation-in-part application of PCT/CN2017/088240, entitled \u201cVIDEO DETECTION METHOD AND SERVER\u201d filed on Jun. 14, 2016, which claims priority to Chinese Patent Application No. 201610457780.1, filed with the State Intellectual Property Office of the People's Republic of China on Jun. 22, 2016, and entitled \u201cVIDEO DETECTION METHOD AND SERVER\u201d, all of which are incorporated herein by reference in their entirety.", + "p0003": "FIELD OF THE TECHNOLOGY", + "p0004": "This application relates to video data processing technologies, and specifically, to a video detection method, a server and a storage medium.", + "p0005": "BACKGROUND OF THE DISCLOSURE", + "p0006": "Video copy detection becomes a hot topic researched by people because of a copyright protection problem of digital videos. In the existing technology, usually, two of the following solutions are used to detect whether a video infringes the copyright: A first solution is to detect audio of the video. However, the same audio may have the copyright of different videos, for example, a music video . The copyright of the audio protects only audio data. Consequently, a copyright protection requirement for video data cannot be met if copyright protection is only performed on audio. The second solution is to detect images of a video. Compared with the audio data, the video data has a large quantity of information. During detection, a large quantity of fingerprint features need to be calculated, so as to be compared with fingerprint features in a copyright library. This needs a strong calculation capability and needs to consume a large quantity of calculation resources.", + "p0007": "SUMMARY", + "p0008": "To resolve a currently-existing technical problem, embodiments of this application provide a video detection method and a server. A video data copyright protection requirement is met, in addition, a strong calculation capability is not needed and calculation resource consumption is reduced.", + "p0009": "An embodiment of this application provides a video detection method, including:", + "p0010": "obtaining, by a server, first video data to be detected, and decoding, by the server, the first video data, to obtain audio data of the first video data;", + "p0011": "analyzing and identifying, by the server, the audio data, to obtain audio fingerprint data corresponding to the audio data; and querying, by the server based on the audio fingerprint data, an audio fingerprint library;", + "p0012": "obtaining, by the server, a video label and a time parameter corresponding to the audio fingerprint data when the audio fingerprint library includes the audio fingerprint data;", + "p0013": "querying, by the server, a video copyright library based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracting a second picture that is in the first video data and that satisfies the time parameter;", + "p0014": "separately extracting, by the server, a first feature parameter of the first picture and a second feature parameter of the second picture; and", + "p0015": "comparing, by the server, the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "p0016": "In the foregoing solution, before the querying, by the server, a video copyright library based on the video label, the method includes:", + "p0017": "performing, by the server, picture extraction on second video data having copyright, and storing the extracted picture, a time parameter corresponding to the picture, and a video label corresponding to the picture in the video copyright library.", + "p0018": "In the foregoing solution, the performing, by the server, picture extraction on second video data having copyright includes:", + "p0019": "performing, by the server, scene identification on the second video data, and identifying and filtering a first picture collection representing scene switching in the second video data, to obtain a second picture collection;", + "p0020": "analyzing and identifying, by the server, a picture in the second picture collection, to obtain edge feature information of the picture in the second picture collection; and", + "p0021": "extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold.", + "p0022": "In the foregoing solution, the extracting, by the server, a picture of which a quantity of the edge feature information reaches a preset threshold includes:", + "p0023": "generating, by the server, the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and", + "p0024": "extracting, by the server, a picture in the third picture collection at a preset time interval.", + "p0025": "In the foregoing solution, the analyzing and identifying, by the server, the audio data, to obtain audio fingerprint data corresponding to the audio data includes:", + "p0026": "extracting, by the server, a feature parameter of the audio data, and obtaining, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "p0027": "In the foregoing solution, the separately extracting, by the server, a first feature parameter of the first picture and a second feature parameter of the second picture includes: separately extracting the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners:", + "p0028": "a scale-invariant feature transform manner and a histogram of oriented gradient manner.", + "p0029": "In the foregoing solution, the comparing, by the server, the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library includes:", + "p0030": "calculating, by the server, similarity between the first feature parameter and the second feature parameter; and determining, by the server, that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "p0031": "An embodiment of this application further provides a server, including: an audio processing module, an audio fingerprint storage module, a video processing module and a video copyright storage module.", + "p0032": "The audio processing module is configured to: obtain first video data to be detected; decode the first video data, to obtain audio data of the first video data; analyze and identify the audio data, to obtain audio fingerprint data corresponding to the audio data; and is further configured to: query, based on the audio fingerprint data, the audio fingerprint storage module; and obtain a video label and a time parameter corresponding to the audio fingerprint data when an audio fingerprint library includes the audio fingerprint data.", + "p0033": "The audio fingerprint storage module is configured to store the audio fingerprint data, and the corresponding video label and time parameter.", + "p0034": "The video processing module is configured to: query the video copyright storage module based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extract a second picture that is in the first video data and that satisfies the time parameter; separately extract a first feature parameter of the first picture and a second feature parameter of the second picture; and compare the first feature parameter with the second feature parameter, and determine, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "p0035": "The video copyright storage module is configured to store the video label, and the corresponding picture and time parameter.", + "p0036": "In the foregoing solution, the video processing module is further configured to: before the video copyright storage module is queried based on the video label, perform picture extraction on second video data having copyright, and store the extracted picture, a time parameter corresponding to the picture, and a video label corresponding to the picture in the video copyright storage module.", + "p0037": "In the foregoing solution, the video processing module is configured to: perform scene identification on the second video data, identify and filter a first picture collection representing scene switching in the second video data, to obtain a second picture collection; analyze and identify a picture in the second picture collection, to obtain edge feature information of the picture in the second picture collection; extract a picture of which a quantity of the edge feature information reaches a preset threshold.", + "p0038": "In the foregoing solution, the video processing module is configured to: generate the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extract a picture in the third picture collection at a preset time interval.", + "p0039": "In the foregoing solution, the audio processing module is configured to extract a feature parameter of the audio data, and obtain, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "p0040": "In the foregoing solution, the video processing module is configured to separately extract the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a SIFT manner and a HOG manner.", + "p0041": "In the foregoing solution, the video processing module is configured to calculate similarity between the first feature parameter and the second feature parameter; and determine that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "p0042": "An embodiment of this application provides a computer storage medium, configured to store computer software instructions used by the foregoing server, and including steps for performing the foregoing video detection method.", + "p0043": "In the video detection method and the server provided by the embodiments of this application, by combining audio content detection and video content detection, and by focusing mainly on audio content detection and subsidiarily on video content detection, calculation capabilities required when only video content detection is used are greatly decreased, calculation resource consumption is reduced, a copyright protection shortcoming by merely using an audio fingerprint is overcome, and a video copyright protection requirement is met.", + "p0044": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0045": "FIG. 1 is a schematic flowchart of a video detection method according to an embodiment of this application;", + "p0046": "FIG. 2 a to FIG. 2 d are schematic diagrams of generating audio fingerprint data according to an embodiment of this application;", + "p0047": "FIG. 3 is a schematic diagram of an extraction matching process of audio fingerprint data according to an embodiment of this application;", + "p0048": "FIG. 4 is a schematic diagram of an establishing process of a video copyright library according to an embodiment of this application;", + "p0049": "FIG. 5 is a schematic flowchart of another video detection method according to an embodiment of this application;", + "p0050": "FIG. 6 a and FIG. 6 b are schematic diagrams of a system architecture applying a video detection method according to an embodiment of this application;", + "p0051": "FIG. 7 is a schematic diagram of a composition structure of a server according to an embodiment of this application; and", + "p0052": "FIG. 8 is a schematic diagram of a composition structure of a server used as hardware according to an embodiment of this application.", + "p0053": "DESCRIPTION OF EMBODIMENTS", + "p0054": "With the development of Internet technologies, there are a growing number of video sharing platforms. A user may upload various video data that includes an MV, a short video, a TV play, a movie, a variety show video, an animation video, and the like. A video sharing platform server performs copyright detection on the video data uploaded by the user. Usually, there is a video copyright database. If content of the uploaded video data is consistent with that of video data in the video copyright database, the uploaded video data has a copyright conflict. Subsequently, an operation is performed on the video data having the copyright conflict or an uploader. For example, the video data having the copyright conflict is deleted, or a legal affair negation is performed with the uploaded under a serious circumstance. If the content of the uploaded video data is not consistent with that of the video data in the video copyright database, it may be determined that the video data is original video data made by the user. Copyright protection in an original and the like may be provided for the user.", + "p0055": "Based on this, the following embodiments of this application are provided. A technical solution of the embodiments of this application describes a process of matching the uploaded video data with the video data in the video copyright database in detail.", + "p0056": "The following describes this application in detail with reference to the accompanying drawings and specific embodiments.", + "p0057": "An embodiment of this application provides a video detection method. FIG. 1 is a schematic flowchart of a video detection method according to an embodiment of this application. As shown in FIG. 1 , the video detection method includes:", + "p0058": "Step 101 : Obtain first video data to be detected, and decode the first video data, to obtain audio data of the first video data.", + "p0059": "Step 102 : Analyze and identify the audio data, to obtain audio fingerprint data corresponding to the audio data.", + "p0060": "Step 103 : Query, based on the audio fingerprint data, an audio fingerprint library.", + "p0061": "Step 104 : Obtain a video label and a time parameter corresponding to the audio fingerprint data when the audio fingerprint library includes the audio fingerprint data.", + "p0062": "Step 105 : Query a video copyright library based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extract a second picture that is in the first video data and that satisfies the time parameter.", + "p0063": "Step 106 : Separately extract a first feature parameter of the first picture and a second feature parameter of the second picture.", + "p0064": "Step 107 : Compare the first feature parameter with the second feature parameter, and determine, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "p0065": "The video detection method of this embodiment is applied to a video sharing platform server or a server cluster.", + "p0066": "In step 101 of this embodiment, the first video data is video data to be detected, which may be specifically video data uploaded by a user, including an MV, a TV play, a movie, an animation video, or the like. The first video data is decoded to obtain the audio data of the first video data.", + "p0067": "In step 102 of this embodiment, the audio data is analyzed and identified to obtain the audio fingerprint data corresponding to the audio data. The audio fingerprint data may be specifically quantized data representing a feature parameter of the audio data, and may be specifically represented by using a binary value. Specifically, the analyzing and identifying the audio data, to obtain audio fingerprint data corresponding to the audio data includes: extracting the feature parameter of the audio data, and obtaining based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "p0068": "In an implementation, audio is a type of sound wave, so that the feature parameter of the audio data may be presented by using a sampling rate. Specifically, FIG. 2 a to FIG. 2 d are schematic diagrams of generating audio fingerprint data according to an embodiment of this application. FIG. 3 is a schematic diagram of an extraction matching process of audio fingerprint data according to an embodiment of this application. As shown by step 201 and step 202 in FIG. 3 , the sampling rate of the audio data is converted to be K samples/s by converting the sampling rate. K is, for example, 8000. The collected audio data represented by one-dimensional time domain is converted into a two-dimensional diagram by using a short-time Fourier transform as shown in FIG. 2 a . A feature point that is obviously extracted in the two-dimensional diagram shown in FIG. 2 a is used as an obvious feature. For example, feature points in FIG. 2 b are obtained by searching peaks in a frequency spectrum. For each selected feature point, for example, a feature point A, as shown in FIG. 2 c , a proper window, for example, a window area 1 in FIG. 2 c , is selected based on time and frequency to perform feature hash value conversion. Specifically, for example, for the feature point A, coordinates of the feature point A are . A feature point with a maximum frequency domain corresponding to a time is selected in the window area 1 , for example, a feature point C. Coordinates of the feature point C are . A hash value formed by the feature point C and the feature point A may be Hash: time= :t 1 , where \u0394t=t 2 \u2212t 1 . In this embodiment, the obtained hash value is represented as a binary value, and the binary data specifically is the audio fingerprint data corresponding to the audio data.", + "p0069": "Certainly, the method for obtaining audio fingerprint data in this embodiment of this application is not limited to the foregoing obtaining method, and another method for obtaining audio fingerprint data that represents an audio data feature and that can be obtained falls within the protection scope of the embodiments of this application.", + "p0070": "In step 103 of this embodiment, the audio fingerprint library ingests audio fingerprint data of video data having copyright, a corresponding video label, and a time parameter corresponding to the audio fingerprint data. The video label may be specifically represented by using a video identification, for example, a sequence number or a code, for convenience of searching in the video copyright library based on the video label. Specifically, the audio fingerprint data in the audio fingerprint library may be obtained by performing audio data extraction on the video data having copyright and extracting according to the method for extracting audio fingerprint data according to this embodiment of this application. No further details are provided herein.", + "p0071": "In this embodiment, when the audio fingerprint library does not include the audio fingerprint data, it indicates that the first video data is not consistent with video data in the video copyright library. That is, it indicates that the first video data and the video data having the copyright do not conflict, and in other words, the first video data does not have a copyright conflict with the video data having the copyright. Correspondingly, step 104 to step 107 are not performed subsequently, and a determining result indicating that the first video data is not consistent with the video data in the video copyright library may be directly obtained.", + "p0072": "As shown by step 203 in FIG. 3 , a video label and a time parameter corresponding to audio fingerprint data are obtained to be used as matching information when an audio fingerprint library includes the audio fingerprint data. The video label may be specifically represented by using a video identification, for example, a sequence number or a code. In an actual application, the audio fingerprint library stores a large quantity of information. To balance a matching time, accelerate a matching speed, and do not reduce accuracy, step 203 uses a fuzzy matching manner. That is, during matching, matching hash amounts having same audio fingerprint data and a same time difference are counted, and the first N pieces of the matched audio fingerprint data are selected to form a potential matched audio segment. In a next stage, each segment of the matched audio fingerprint data is selected to analyze the density of a hash amount of the audio fingerprint data at a particular time. If the density is greater than a threshold, the segment is remained; otherwise, the segment is removed. A hash value segment whose density is greater than the threshold is remained. In this process, a potential audio segment having a low matching density may be removed, thereby improving the matching accuracy. De-duplication selection is performed on the selected potential matching audio segment, and for repeated audio matched in time, audio having a largest hash density is selected as final matching audio. During matching, one-to-one or one-to-many matching may be allowed. In this process, filtering may be performed when the hash density is analyzed, and only a segment of audio having the longest matching time is remained. The foregoing process may be referred to as a classification and filtering process, that is, step 204 . Therefore, final matching information is obtained.", + "p0073": "In step 105 of this embodiment, after the video label corresponding to the audio fingerprint data is obtained, the video copyright library is queried based on the video label. The video copyright library ingests a key frame picture of video data having copyright, the video label, a time parameter corresponding to the key frame picture and the like. The video copyright library is queried to obtain the first picture that is corresponding to the video label and that satisfies the time parameter, the second picture that is in the first video data and that satisfies the time parameter is extracted. Specifically, in a matching process of specific picture content of video data, the video copyright library is queried according to the video label obtained by matching the audio fingerprint data, to obtain a picture collection corresponding to the video label. A picture satisfying the time parameter is read from the picture collection according to the time parameter obtained by matching the audio fingerprint data. Correspondingly, for the first video data, a picture that is in the first video data and that satisfies the time parameter is extracted according to the time parameter obtained by matching the audio fingerprint data.", + "p0074": "In this embodiment, the first feature parameter of the first picture and the second feature parameter of the second picture are separately extracted. The first feature parameter is compared with the second feature parameter to determine whether the first video data is consistent with video data in a copyright database. Specifically, the separately extracting a first feature parameter of the first picture and a second feature parameter of the second picture includes: separately extracting the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a SIFT manner and a HOG manner. Further, the comparing the first feature parameter with the second feature parameter, and determining, based on a comparison result, whether the first video data is consistent with a video in the video copyright library includes: calculating similarity between the first feature parameter and the second feature parameter by using the SIFT manner or the HOG manner; and determining that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "p0075": "Based on an implementation of this embodiment, before the video copyright library is queried based on the video label, in other words, in a process of establishing the video copyright library, the method includes: performing picture extraction on second video data having copyright, and storing the extracted picture, a time parameter corresponding to the picture, and a video label corresponding to the picture in the video copyright library.", + "p0076": "The performing scene identification on second video data having copyright includes: performing scene identification on the second video data, identifying and filtering a first picture collection representing scene switching in the second video data, to obtain a second picture collection; analyzing and identifying a picture in the second picture collection, to obtain edge feature information of the picture; extracting a picture of which a quantity of the edge feature information reaches a preset threshold.", + "p0077": "Specifically, in the process of establishing the video copyright library, key frame picture extraction is performed on all video data having copyright. FIG. 4 is a schematic diagram of an establishing process of a video copyright library according to an embodiment of this application. As shown in FIG. 4 , the video data having copyright is input and a key frame picture of the input video data is extracted. A process of extracting the key frame picture of the video data includes: first, performing scene switching detection on the video data. The scene switching detection may be performed by detecting a foreground and/or a background in the picture. When it is detected that the foregrounds and/or backgrounds of two pictures are inconsistent, it may be determined that a time point between corresponding time points of the two pictures is a scene switching time point. It may also be understood that a first picture having a former time is the last picture of the previous scene, and the other picture having a later time is the first picture of a next scene. In this embodiment, scene identification is performed on the input video data, and a first picture collection representing scene switching is identified. In the process of extracting the key frame picture, first, the first picture collection is filtered from a picture collection included by the video data, to prevent that an extracted key frame picture is at a scene switching position and accuracy of subsequent picture content matching is reduced. Further, analysis and identification are further performed on remaining pictures in a remaining second picture collection. Specifically, analysis and identification are performed on complexity of the pictures, to extract a picture with high complexity as a key frame picture. In an implementation, this embodiment of this application may use analysis on picture edge feature information to find out, in the second picture collection, a picture that has a large quantity of the edge feature information. A larger quantity of the edge feature information of a picture indicates higher complexity of the picture. In a process of matching two pictures, more complex content indicates higher matching accuracy.", + "p0078": "In an implementation, the extracting a picture of which a quantity of the edge feature information reaches a preset threshold includes: generating the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extracting a picture in the third picture collection at a preset time interval.", + "p0079": "Specifically, in this embodiment, there may be many pictures of which the quantity of the edge feature information is large. To reduce picture storage space and a calculation amount of feature matching, a picture may be extracted as a key frame picture at the preset time interval. For example, a picture is extracted every K second. This may greatly reduce a quantity of the extracted key frame pictures, greatly reduce a picture storage quantity and the calculation amount of feature matching, and reduce calculation resource consumption of a server.", + "p0080": "Based on the video detection solution described above, in short, the technical solution of this embodiment of this application focuses mainly on audio detection and subsidiarily on video detection. In other words, detection is performed by combining audio content and video content to determine whether a video to be detected and the copyright conflict. Specifically, FIG. 5 is a schematic flowchart of another video detection method according to an embodiment of this application. As shown in FIG. 5 , first, audio data is extracted from video data to be detected. First, it is matched whether the audio data is consistent with audio data of video data having copyright. Specifically, audio fingerprint data of the audio data is matched with audio fingerprint data in an audio fingerprint library. If the audio fingerprint data of the audio data does not match the audio fingerprint data in the audio fingerprint library, it is determined that a matching result is that the video to be detected and the video having the copyright do not conflict. If the audio fingerprint data of the audio data matches the audio fingerprint data in the audio fingerprint library, a video label and a time parameter corresponding to the audio fingerprint data are obtained from the audio fingerprint library. Content of the video to be detected is further matched with content of the video in a video copyright library. Specifically, the video copyright library is queried based on the video label to obtain a video source. A first picture satisfying the time parameter is obtained from the video source. A picture is extracted from the video to be detected according to the time parameter to be used as a second picture. Fingerprint feature extraction is separately performed on the first picture and the second picture. Specifically, feature extraction may be performed according to a SIFT manner. In addition, feature matching is performed, and similarity between the two pictures is calculated. When the similarity obtained through calculation reaches a preset threshold, it indicates that the similarity between the two pictures is extremely high, and it may be determined that a matching result is that the video to be detected and the video having copyright conflict. When the similarity obtained through calculation does not reach a preset threshold, it indicates that the similarity between the two pictures is low, and it may be determined that the matching result is that the video to be detected and the video having copyright do not conflict.", + "p0081": "When the matching result is that the video to be detected and the video having copyright do not conflict, it may be determined that the video data is original video data made by a user and copyright protection in an original and sharing of advertising revenue may be provided for the user, to encourage excellent video data makers, thereby providing more excellent video content for a video sharing platform. When the matching result is that the video to be detected and the video having copyright conflict, the video data to be detected has a copyright conflict. Subsequently, an operation is performed on the video data having the copyright conflict or an uploader. For example, the video data having the copyright conflict is deleted, or a legal affair negation is performed with the uploaded under a serious circumstance. Copyright protection is provided for video data having copyright to a great extent.", + "p0082": "In the existing technology, an detection solution based on video content usually includes two of the following: 1. a space color-based video fingerprint; 2. a feature extraction-based video fingerprint.", + "p0083": "The space color-based video fingerprint basically is a histogram of a picture in a time period in a particular area. A feature of a color varies based on different formats of a video. Consequently, the color-based video fingerprint does not have a high anti-nose capability, for example, changes such as adding a brand and a black border.", + "p0084": "A two-dimensional discrete cosine transform video fingerprint is representative of the feature-based video fingerprint. The 2D-DCT video fingerprint is widely applied to hash-based image retrieval. Main steps of the hash-based image retrieval include: First, frame rate transforming is performed on a video at time, and the frame rate is changed into a low frame rate F . Subsequently, pictures are scaled down and then are changed into black and white pictures. The changed black and white pictures form a small segment in a time domain of the video in a time length. Subsequently, TIRI is used to combine information of the time domain of continuous images of each slice, to obtain a picture. Next, the 2D-DCT is performed on the obtained picture representing the time of the time domain . A median value is found in the transformed DCT values. A transformed fingerprint is changed into a two-dimensional vector by using a two-dimensional median value, thereby representing a feature of the video in an area.", + "p0085": "The detection solution for space color-based video content has unfavorable accuracy and anti-nose capabilities. The detection solution for feature-based video content needs a strong calculation capability and consumes a large quantity of calculation resources. Compared with the existing detection solution for video content, the video detection solution of this embodiment of this application focuses mainly on audio fingerprint detection and subsidiarily on video fingerprint detection. This greatly decreases calculation capabilities required when only video content detection is used, reduces calculation resource consumption, overcomes a copyright protection shortcoming by merely using an audio fingerprint, and meets a video copyright protection requirement.", + "p0086": "FIG. 6 a and FIG. 6 b are schematic diagrams of a system architecture applying a video detection method according to an embodiment of this application. As shown in FIG. 6 a , servers 31 , . . . , 3 n and terminal devices 21 - 23 are included. The terminal devices 21 - 23 may interact with the server through a network. The terminal devices 21 - 23 may specifically include a mobile phone, a desktop computer, a notebook computer, and the like. The servers 31 , . . . , 3 n may be specifically a server cluster of a video sharing platform, and the server cluster may be implemented by using FIG. 6 b . As shown in FIG. 6 b , the server cluster include a task server 311 , an audio detection server 312 , a video detection server 313 , and a content storage server 314 . Specifically, after the terminal device uploads video data, the task server 311 obtains the video data and initiates an detection task. The task is submitted to the audio detection server 312 to perform audio detection and is compared with current content having an audio copyright. Videos that have copyright and that have same audio are found by using the audio detection server 312 and matching information is obtained. Further, the matching information is submitted to the video detection server 313 . Subsequently, video content detection is performed on the input video and these videos that have copyright and that have the same audio, thereby determining whether audio and the video of the current video are the same as the videos that have copyright and that have the same audio. A determining result is stored in the content storage server 314 . Finally, a related worker may perform a series of work related to the copyright based on content stored in the content storage server 314 , for example, a legal affair negation.", + "p0087": "An embodiment of this application further provides a server. FIG. 7 is a schematic diagram of a composition structure of a server according to an embodiment of this application. As shown in FIG. 7 , the server includes: an audio processing module 41 , an audio fingerprint storage module 42 , a video processing module 43 , and a video copyright storage module 44 .", + "p0088": "The audio processing module 41 is configured to: obtain first video data to be detected; decode the first video data, to obtain audio data of the first video data; analyze and identify the audio data, to obtain audio fingerprint data corresponding to the audio data; and is further configured to: query, based on the audio fingerprint data, the audio fingerprint storage module 42 ; and obtain a video label and a time parameter corresponding to the audio fingerprint data when an audio fingerprint library includes the audio fingerprint data.", + "p0089": "The audio fingerprint storage module 42 is configured to store the audio fingerprint data, and the corresponding video label and time parameter.", + "p0090": "The video processing module 43 is configured to: query the video copyright storage module 44 based on the video label, to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extract a second picture that is in the first video data and that satisfies the time parameter; separately extract a first feature parameter of the first picture and a second feature parameter of the second picture; and compare the first feature parameter with the second feature parameter, and determine, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "p0091": "The video copyright storage module 44 is configured to store the video label, and the corresponding picture and time parameter.", + "p0092": "The server in this embodiment may be an independent server or a server cluster. Based on the system architecture shown in FIG. 6 a and FIG. 6 b , modules in this embodiment may be implemented by using any server or server cluster in the system architecture. The server cluster may be servers shown in FIG. 6 b.", + "p0093": "In this embodiment, the first video data is video data to be detected, which may be specifically video data uploaded by a user, including an MV, a TV play, a movie, an animation video, or the like. The audio processing module 41 decodes the obtained first video data to separate the audio data of the first video data.", + "p0094": "In this embodiment, the audio processing module 41 analyzes and identifies the audio data to obtain the audio fingerprint data corresponding to the audio data. The audio fingerprint data may be specifically quantized data representing a feature parameter of the audio data, and may be specifically represented by using a binary value. Specifically, the audio processing module 41 is configured to extract a feature parameter of the audio data, and obtain, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "p0095": "In an implementation, audio is a type of sound wave, so that the feature parameter of the audio data may be presented by using a sampling rate. Specifically, as shown in FIG. 3 , the sampling rate of the audio data is converted to be K samples/s by converting the sampling rate. K is, for example, 8000. The collected audio data represented by one-dimensional time domain is converted into a two-dimensional diagram by using a short-time Fourier transform, as shown in FIG. 2 a . A feature point that is obviously extracted in the two-dimensional diagram shown in FIG. 2 a is used as an obvious feature. For example, feature points in FIG. 2 b are obtained by searching peaks in a frequency spectrum. For each selected feature point, for example, a feature point A, as shown in FIG. 2 c , a proper window, for example, a window area 1 in FIG. 2 c , is selected based on time and frequency to perform feature hash value conversion. Specifically, for example, for the feature point A, coordinates of the feature point A are . A feature point with a maximum frequency domain corresponding to a time is selected in the window area 1 , for example, a feature point C. Coordinates of the feature point C are . A hash value formed by the feature point C and the feature point A may be Hash: time= :t 1 , where \u0394t=t 2 \u2212t 1 . In this embodiment, the obtained hash value is represented as a binary value, and the binary data specifically is the audio fingerprint data corresponding to the audio data.", + "p0096": "Certainly, the method for obtaining the audio fingerprint data by the audio processing module 41 in this embodiment of this application is not limited to the foregoing obtaining method, and another method for obtaining audio fingerprint data that represents an audio data feature and that can be obtained falls within the protection scope of the embodiments of this application.", + "p0097": "In this embodiment, the audio fingerprint storage module 42 ingests audio fingerprint data of video data having a copyright, a corresponding video label, and a time parameter corresponding to the audio fingerprint data. The video label may be specifically represented by using a video identification, for example, a sequence number or a code, for convenience of searching in the video copyright storage module 44 based on the video label. Specifically, the audio fingerprint data in the audio fingerprint storage module 42 may be obtained by performing audio data extraction on the video data having copyright by the audio processing module 41 and extracting according to the method for extracting audio fingerprint data according to this embodiment of this application. Then the audio fingerprint data is stored in the audio fingerprint storage module 42 . For the specific obtaining manner, no further details are provided herein.", + "p0098": "In this embodiment, when the audio fingerprint storage module 42 does not include the audio fingerprint data, it indicates that the first video data is not consistent with video data in the video copyright storage module 44 . That is, it indicates that the first video data and the video data having the copyright do not conflict, and in other words, the first video data does not have a copyright conflict with the video data having the copyright. Correspondingly, a process of matching the video content is not performed subsequently, and a determining result indicating that the first video data is not consistent with the video data in the video copyright storage module 44 may be directly obtained.", + "p0099": "As shown in FIG. 3 , when the audio fingerprint storage module 42 includes the audio fingerprint data, the audio processing module 41 obtains a video label and a time parameter corresponding to the audio fingerprint data. The video label may be specifically represented by using a video identification, for example, a sequence number or a code. In an actual application, the audio fingerprint library stores a large quantity of information. To balance a matching time, accelerate a matching speed, and do not reduce accuracy, a fuzzy matching manner may be used. That is, during matching, matching hash amounts having same audio fingerprint data and a same time difference are counted, and the first N pieces of the matched audio fingerprint data are selected to form a potential matched audio segment. In a next stage, each segment of the matched audio fingerprint data is selected to analyze the density of a hash amount of the audio fingerprint data at a particular time. If the density is greater than a threshold, the segment is remained; otherwise, the segment is removed. A hash value segment whose density is greater than the threshold is remained. In this process, a potential audio segment having a low matching density may be removed, thereby improving the matching accuracy. De-duplication selection is performed on the selected potential matching audio segment, and for repeated audio matched in time, audio having a largest hash density is selected as final matching audio. During matching, one-to-one or one-to-many matching may be allowed. In this process, filtering may be performed when the hash density is analyzed, and only a segment of audio having the longest matching time is remained. The foregoing process may be referred to as a classification and filtering process. Therefore, final matching information is obtained.", + "p0100": "In this embodiment, after the audio processing module 41 obtains the video label corresponding to the audio fingerprint data, the video label and the corresponding time parameter are sent to the video processing module 43 . The video processing module 43 queries the video copyright storage module 44 based on the video label. The video copyright storage module 44 ingests a key frame picture of video data having copyright, a video label, a time parameter corresponding to the key frame picture, and the like. The video processing module 43 queries the video copyright storage module 44 , to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracts a second picture that is in the first video data and that satisfies the time parameter. Specifically, in a matching process of specific picture content of video data, the video copyright storage module 44 is queried according to the video label obtained by matching the audio fingerprint data, to obtain a picture collection corresponding to the video label. A picture satisfying the time parameter is read from the picture collection according to the time parameter obtained by matching the audio fingerprint data. Correspondingly, for the first video data, a picture that is in the first video data and that satisfies the time parameter is extracted according to the time parameter obtained by matching the audio fingerprint data.", + "p0101": "In this embodiment, the first feature parameter of the first picture and the second feature parameter of the second picture are separately extracted. The first feature parameter is compared with the second feature parameter to determine whether the first video data is consistent with video data in a copyright database. Specifically, the video processing module 43 is configured to separately extract the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a SIFT manner and a HOG manner. Further, the video processing module 43 is configured to calculate similarity between the first feature parameter and the second feature parameter by using the SIFT manner or the HOG manner; and determine that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "p0102": "Based on an implementation of this embodiment, before the video copyright storage module 44 is queried based on the video label, in other words, in a process of establishing the video copyright storage module 44 , the video processing module 43 is further configured to: perform picture extraction on second video data having copyright, and storing the extracted picture, a time parameter corresponding to the picture, and a video label corresponding to the picture in the video copyright storage module 44 .", + "p0103": "The video processing module 43 is configured to: perform scene identification on the second video data, identify and filter a first picture collection representing scene switching in the second video data, to obtain a second picture collection; analyze and identify a picture in the second picture collection, to obtain edge feature information of the picture; extract a picture of which a quantity of the edge feature information reaches a preset threshold.", + "p0104": "Specifically, in the process of establishing the video copyright storage module 44 , key frame picture extraction is performed on all video data having copyright. As shown in FIG. 4 , the video data having copyright is input and the video processing module 43 extracts a key frame picture of the input video data. A process of extracting the key frame picture of the video data includes: first, performing scene switching detection on the video data. The scene switching detection may be performed by detecting a foreground and/or a background in the picture. When it is detected that the foregrounds and/or backgrounds of two pictures are inconsistent, it may be determined that a time point between corresponding time points of the two pictures is a scene switching time point. It may also be understood that a first picture having a former time is the last picture of the previous scene, and the other picture having a later time is the first picture of a next scene. In this embodiment, scene identification is performed on the input video data, and a first picture collection representing scene switching is identified. In the process of extracting the key frame picture, first, the first picture collection is filtered from a picture collection included by the video data, to prevent that an extracted key frame picture is at a scene switching position and accuracy of subsequent picture content matching is reduced. Further, the video processing module 43 further analyzes and identifies remaining pictures in a remaining second picture collection. Specifically, analysis and identification are performed on complexity of the pictures, to extract a picture with high complexity as a key frame picture. In an implementation, this embodiment of this application may use analysis on picture edge feature information to find out, in the second picture collection, a picture that has a large quantity of the edge feature information. A larger quantity of the edge feature information of a picture indicates higher complexity of the picture. In a process of matching two pictures, more complex content indicates higher matching accuracy.", + "p0105": "In an implementation, the video processing module 43 is configured to: generate the picture of which the quantity of the edge feature information reaches the preset threshold into a third picture collection; and extract a picture in the third picture collection at a preset time interval.", + "p0106": "Specifically, in this embodiment, there may be many pictures of which the quantity of the edge feature information is large. To reduce picture storage space and a calculation amount of feature matching, the video processing module 43 may extract a picture as a key frame picture at the preset time interval. For example, a picture is extracted every K second. This may greatly reduce a quantity of the extracted key frame pictures, greatly reduce a picture storage quantity and the calculation amount of feature matching, and reduce calculation resource consumption of a server.", + "p0107": "In this embodiment, in an actual application, the audio processing module 41 and the video processing module 43 may be implemented by a central processing unit , a digital signal processor , a microcontroller unit , or a field-programmable gate array . In an actual application, the audio fingerprint storage module 42 and the video copyright storage module 44 may be implemented by a memory in the server.", + "p0108": "In the technical solution of this embodiment of this application, by combining audio content detection and video content detection, and by focusing mainly on audio content detection and subsidiarily on video content detection, calculation capabilities required when only video content detection is used are greatly decreased, calculation resource consumption is reduced, a copyright protection shortcoming by merely using an audio fingerprint is overcome, and a video copyright protection requirement is met.", + "p0109": "An example indicating that the server in this embodiment serves as a hardware entity is shown in FIG. 8 . The apparatus includes a processor 61 , a storage medium 62 , and at least one external communications interface 63 . The processor 61 , the storage medium 62 , and the external communications interface 63 are connected to each other by using a bus 64 .", + "p0110": "It needs to be noted that, the foregoing descriptions related to the server item are similar to the foregoing method descriptions. Descriptions of a beneficial effect of a same method are not further described. For technical details that are not disclosed in the server embodiments of this application, refer to the method embodiments of this application.", + "p0111": "A person skilled in the art should understand that the embodiments of this application may be provided as a method, a system, or a computer program product. Therefore, this application may use a form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware. Moreover, this application may use a form of a computer program product that is implemented on one or more computer-usable storage media that include computer-usable program code.", + "p0112": "This application is described with reference to flowcharts and/or block diagrams of the method, the device , and the computer program product in the embodiments of this application. It should be understood that computer program instructions may be used to implement each process and/or each block in the flowcharts and/or the block diagrams and a combination of a process and/or a block in the flowcharts and/or the block diagrams. These computer program instructions may be provided for a general-purpose computer, a dedicated computer, an embedded processor, or a processor of any other programmable data processing device to generate a machine, so that the instructions executed by a computer or a processor of any other programmable data processing device generate an apparatus for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.", + "p0113": "These computer program instructions may also be stored in a computer-readable memory that can instruct the computer or any other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate an artifact that includes an instruction apparatus. The instruction apparatus implements a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.", + "p0114": "These computer program instructions may also be loaded onto a computer or another programmable data processing device, so that a series of operations and steps are performed on the computer or the another programmable device, thereby generating computer-implemented processing. Therefore, the instructions executed on the computer or the another programmable device provide steps for implementing a specific function in one or more processes in the flowcharts and/or in one or more blocks in the block diagrams.", + "p0115": "The foregoing descriptions are merely preferred embodiments of this application, but are not used to limit the protection scope of this application.", + "features": { + "F1": "A server obtains first video data to be detected and decodes the first video data to obtain audio data of the first video data.", + "F2": "The server analyzes and identifies the audio data to obtain audio fingerprint data corresponding to the audio data.", + "F3": "The server extracts a picture corresponding to a predefined scene switching in second video data having copyright, performs scene identification on the second video data, and identifies and filters a first picture collection representing the predefined scene switching in the second video data to obtain a second picture collection.", + "F4": "The server analyzes and identifies a picture in the second picture collection to obtain edge feature information of the picture in the second picture collection.", + "F5": "The server extracts a picture of which a quantity of the edge feature information reaches a preset threshold and stores the extracted picture, a time parameter corresponding to the extracted picture, and a video label corresponding to the extracted picture in a video copyright library.", + "F6": "The server queries based on the audio fingerprint data, an audio fingerprint library that stores audio fingerprint data of copyrighted video data and video labels and time parameters corresponding to the audio fingerprint data.", + "F7": "In accordance with a determination that the audio fingerprint data exists in the audio fingerprint library, the server obtains a video label and a time parameter corresponding to the audio fingerprint data from the audio fingerprint library.", + "F8": "The server queries the video copyright library based on the video label to obtain a first picture that is corresponding to the video label and that satisfies the time parameter, and extracts a second picture that is in the first video data and that satisfies the time parameter.", + "F9": "The server separately extracts a first feature parameter of the first picture and a second feature parameter of the second picture.", + "F10": "The server compares the first feature parameter with the second feature parameter and determines, based on a comparison result, whether the first video data is consistent with a video in the video copyright library.", + "F11": "For the picture of which a quantity of the edge feature information reaches a preset threshold, the server generates this picture into a third picture collection and extracts a picture in the third picture collection at a preset time interval.", + "F12": "The server extracts a feature parameter of the audio data and obtains, based on the feature parameter, the audio fingerprint data corresponding to the audio data.", + "F13": "The server separately extracts the first feature parameter of the first picture and the second feature parameter of the second picture according to at least one of the following manners: a scale-invariant feature transform SIFT manner and a histogram of oriented gradient HOG manner.", + "F14": "The server calculates similarity between the first feature parameter and the second feature parameter and determines that the first video data is consistent with the video in the video copyright library when the similarity reaches a preset threshold.", + "F15": "The audio fingerprint data is quantized data representing a feature parameter of the audio data.", + "F16": "A server comprises at least one processor and a memory storing a processor-executable instruction, where the instruction, when executed by the processor, causes the server to perform the operations of the method described." + } + } + }, + { + "FAN": "76109416", + "CPC": [ + "A61B-008/08/5", + "A61B-008/12", + "A61B-008/481", + "A61B-008/483", + "A61B-008/485", + "A61B-008/488", + "A61B-008/5223", + "A61B-008/5246", + "G06F-016/00", + "G06F-018/2148", + "G06F-018/2415", + "G06F-018/2431", + "G06K-009/46", + "G06K-009/62/57", + "G06K-009/62/77", + "G06K-009/62/8", + "G06K-2209/051", + "G06T-007/00/12", + "G06T-007/00/14", + "G06T-007/11", + "G06T-2207/10132", + "G06T-2207/20076", + "G06T-2207/20081", + "G06T-2207/20084", + "G06T-2207/30081", + "G06T-2207/30096", + "G06V-2201/031", + "G16H-030/40", + "G16H-040/63", + "G16H-050/20", + "G16H-050/70" + ], + "Content": { + "title": "System comprising indicator features in high-resolution micro-ultrasound images", + "pa01": "This invention provides a system and method for providing Indicator Features in high resolution micro-ultrasound prostate images, wherein the ultrasound device operates at a center frequency of 15 MHz and above. Indicator Features are features, which have been identified alone and/or in combination with other features in high resolution micro-ultrasound images, and have been determined to be significantly statistically correlated to either benign tissue or some grade of cancerous tissue on the basis of predictive probabilities. These Indicator Features can be used to train a linear or non-linear classifier, which can be used to classify patient images of the prostate. These Indicator Features can be used to guide a clinician as to where to take a biopsy core from the prostate, as well as assist in diagnosis of the tissue.", + "c-en-0001": "A method for generating one or more Indicator Features in high resolution ultrasound images, which can be used in assigning risk of prostate cancer during high resolution micro-ultrasound imaging the method comprising the steps of: i) generating a list of Possible Features in the images by: a) obtaining a high resolution micro-ultrasound image of prostate tissue and surrounding tissue that contains biopsied prostate tissue, the biopsied prostate tissue having also been biopsied and graded for a stage of cancer, ranging from benign to the highest grade; b) segmenting a region of the image corresponding to the biopsied prostate tissue on the basis of contrasting areas or groups of pixels in the image, wherein each area or group of pixels constitutes a detectable feature; and c) characterizing and providing a unique label to all detectable features and/or combinations of detectable features only in the region of the image that corresponds to biopsied prostate tissue, the detectable features and/or combinations of detectable features with the unique labels comprising the list of Possible Features; ii) repeating steps a) through c) until all detectable recurring patterns have been included in the list of Possible Features; iii) generating a Table of Candidate Features by: d) reading the micro-ultrasound images in an area corresponding to graded prostate tissue which has been graded for a stage of cancer but for which a result for the stage of cancers is not known to a reader so as to identify Candidate Features and/or combinations of Candidate Features, wherein the Candidate Features and/or combinations of Candidate Features are a subset of the list of Possible Features; e) recording in the Table of Candidate Features the Candidate Features and/or combinations of Candidate Features observed in the area corresponding to graded prostate tissue; iv) eliminating non-predictive features from the Table of Candidate Features by: f) performing at least one of: 1) correlating in the Table of Candidate Features the stage of cancer in the biopsied prostate tissue that corresponds to the area in the image where each of the Candidate Features and/or combinations of Candidate Features are identified, wherein the stage ranges from benign to the highest grade, wherein the correlation is a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; and 2) using the Table of Candidate Features as a training set of features to train a linear or nonlinear feature elimination classifier to classify a Candidate Feature or combination of Candidate Features as cancerous or non-cancerous and thereby determining a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; and g) eliminating, from the Table of Candidate Features for generating the Indicator Features, Candidate Features or combinations of Candidate Features that have a predicted probability of cancer that is below a threshold value, the predicted probability of cancer being determined in step f).", + "c-en-0002": "The method of claim 1 , wherein the high resolution micro-ultrasound image is generated using an ultrasonic scanning device that can supply ultrasound at a center frequency of 15 MHz to 70 MHz.", + "c-en-0003": "The method of claim 1 , wherein the image is any one or a combination of: B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhancement, or Spectral Analysis.", + "c-en-0004": "The method of claim 1 , wherein the reading is conducted by: i) a human; ii) a computer; or iii) a human and a computer.", + "c-en-0005": "The method of claim 1 wherein the one or more generated Indicator Features comprise: i) small regular ducts \u201cSwiss cheese;\u201d ii) regular ductal pattern \u201cSwiss cheese\u201d; iii) hyperechoic with ductal patches; iv) hyperechoic with or without ductal patches; v) mild heterogeneity with small bright echoes in hyperechoic tissue; vi) mild heterogeneity; vii) bright echoes in hyperechoic tissue; viii) heterogeneous \u201csmudged/mottled\u201d; ix) heterogeneous \u201ccauliflower\u201d; x) heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance; xi) bright echoes; xii) irregular prostate ; xiii) irregular prostate border; xiv) mixed-echo lesions; xv) irregular shadowing; or xvi) mixed echo lesion causing irregular prostate borders.", + "c-en-0006": "The method of claim 1 , wherein additional patient data is incorporated in the Table of Candidate Features and correlated with each Candidate Feature or combination of Candidate Features to the stage of cancer present in the tissue to generate an Augmented Table of Candidate Features, which is then used to train a linear or non-linear classifier at step f).", + "c-en-0007": "The method of claim 1 , including additional steps for training a classifier, the method comprising the steps of: i) providing a Training Set of Indicator Features comprising the Indicator Features and/or combinations of the Indicator Features correlated to pathology of the tissue; ii) using the Training Set of Indicator Features to train a linear or nonlinear classifier that classifies Indicator Features and/or combinations of Indicator Features as benign or cancerous and thereby determine a predicted probability of cancer for an Indicator Feature and/or combinations of Indicator Features.", + "c-en-0008": "The method of claim 7 , additionally comprising the step of including additional patient data into the Training Set of Indicator Features, wherein the additional patient data is correlated to an Indicator Feature or combination of Indicator Features.", + "c-en-0009": "The method of claim 7 , additionally comprising steps to construct a multiclass classifier, wherein the additional steps are: i) grouping the Indicator Features and combinations of Indicator Features based on the predicted probabilities and confidence intervals determined by the classifier; and ii) forming a new multiclass classifier by assigning a risk level to each grouping of Indicator Features, based on the desired output scale of the classifier.", + "c-en-0010": "A program storage device readable by a computer tangibly embodying a program of instructions executable by the computer to perform the method steps for generating Indicator Features in high resolution micro-ultrasound prostate images, the method comprising the steps of: i) generating a list of Possible Features in the images by: a. obtaining a high resolution micro-ultrasound image of prostate tissue and surrounding tissue that contains biopsied prostate tissue, the biopsied prostate tissue having also been biopsied and graded for a stage of cancer, ranging from benign to the highest grade; b. segmenting a region of the image corresponding to the biopsied prostate tissue on the basis of contrasting areas in the image, wherein each area constitutes a detectable feature; and c. characterizing and providing a unique label to all detectable features and/or combinations of detectable features only in the region of the image that corresponds to biopsied prostate tissue, the detectable features and/or combinations of detectable features with the unique labels comprising the list of Possible Features; ii) repeating steps a. through c. until all detectable recurring patterns have been included in the list of Possible Features; iii) generating a Table of Candidate Features by: d. reading the micro-ultrasound images in an area corresponding to graded prostate tissue which has been graded for a stage of cancer but for which a result for the stage of cancers is not known to a reader so as to identify Candidate Features and/or combinations of Candidate Features, wherein the Candidate Features and/or combinations of Candidate Features are a subset of the list of Possible Features; e. recording in the Table of Candidate Features the Candidate Features and/or combinations of Candidate Features observed in the area corresponding to graded prostate tissue; iv) eliminating non-predictive features from the Table of Candidate Features by: f) performing at least one of: 1) correlating in the Table of Candidate Features the stage of cancer in the biopsied prostate tissue that corresponds to the area in the image where each of the Candidate Features and/or combinations of Candidate Features are identified, wherein the stage ranges from benign to the highest grade, wherein the correlation is a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; and 2) using the Table of Candidate Features as a training set of features to train a linear or nonlinear feature elimination classifier to classify a Candidate Feature or combination of Candidate Features as cancerous or non-cancerous and thereby determining a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; and g. eliminating, from the Table of Candidate Features for generating the Indicator Features, Candidate Features or combinations of Candidate Features that have a predicted probability of cancer that is below a threshold value, the predicted probability of cancer being determined in step f).", + "c-en-0011": "The program storage device of claim 10 , wherein the image is any one or a combination of: B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhancement, or Spectral Analysis.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is the National Stage of International Patent Application No. PCT/CA2016/051313, filed Nov. 10, 2016, which claims priority from U.S. Patent Application No. 62/253,310, filed Nov. 10, 2015, the disclosures of each of which are incorporated herein by reference in their entireties for any and all purposes.", + "p0003": "FIELD OF THE INVENTION", + "p0004": "The present disclosure relates generally to medical imaging and diagnostics, and more specifically to Indicator Features in high resolution micro-ultrasound images of the prostate.", + "p0005": "SUMMARY", + "p0006": "This invention provides a system and method for providing Indicator Features in high resolution micro-ultrasound prostate images, wherein the ultrasound device operates at a center frequency of 15 MHz and above. These Indicator Features can be used to train a linear or non-linear classifier, which can be used to classify patient images of the prostate. These Indicator Features can be used to guide a clinician as to where to take a biopsy core from the prostate, as well as assist in diagnosis of the tissue.", + "p0007": "COPYRIGHT AND TRADEMARK NOTICE", + "p0008": "A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. Trademarks are the property of their respective owners.", + "p0009": "BACKGROUND TO THE INVENTION", + "p0010": "The prostate is a walnut sized gland found beneath the bladder and in front of the rectum that surrounds part of the male urethra. The prostate goes through two main periods of growth. In early puberty, the prostate doubles in size, and then, around age 25, the prostate begins to grow again and continues to grow throughout most of a man's life. The continuing enlargement of the prostate does not usually cause problems until later in life. There are three main disorders of the prostate, which can be reflected in ultrasound images: Benign Prostatic Hyperplasia ; Prostatitis; and Prostatic Cancer.", + "p0011": "Benign prostatic hypertrophy or hyperplasia is one of the most common medical problems experienced by men over 50 years old. Hyperplastic enlargement of the prostate gland, or enlargement due to abnormal but benign multiplication of the cells thereof, often leads to compression of the urethra thereby resulting in obstruction of the urinary tract. Benign prostatic hyperplasia is due to a combination of stromal and glandular hyperplasia, predominantly of the transitional zone .", + "p0012": "Unlike BPH, which occurs primarily in older men, prostatitis can occur in both younger and older men , with the median reported patient age at about 40 years of age. There are several classifications or types of prostatitis, each of which may have different characteristics, manifestations, symptoms, or treatment protocols. Type I is acute bacterial prostatitis; Type II is chronic bacterial prostatitis; Type III is chronic prostatitis and/or chronic pelvic pain syndrome ; and Type IV is asymptomatic inflammatory prostatitis.", + "p0013": "Prostate cancer is the most prevalent newly diagnosed malignancy in men, second only to lung cancer in causing cancer-related deaths. Clinically localized disease is usually suspected based on an elevated prostate specific antigen test or abnormal digital rectal exam , prompting transrectal ultrasound guided biopsy of the prostate for definitive diagnosis. TRUS however, is not reliable enough to be used solely as a template for biopsy. There are cancers that are not visible on TRUS. Furthermore, in PSA screened populations, the accuracy of TRUS was only 52% due to false-positive findings encountered. Increased tumor vessels have been shown microscopically in prostate cancer compared with benign prostate tissue.", + "p0014": "Efficacy of color and power Doppler ultrasound for targeting cancer has not been demonstrated, probably due to limited resolution and small flow velocities. Elasticity imaging, with its many variants, is a new modality that is currently under extensive investigation. It is evident that given the limitations of the present diagnostic protocols, development of a new imaging modality that improves visualization and biopsy yield of prostate cancer would be beneficial.", + "p0015": "Appropriate imaging of prostate cancer is a crucial component for diagnosing prostate cancer and its staging, in addition to PSA levels and DRE. The current state of prostate imaging for diagnosis of prostate cancer includes ultrasound, ultrasound-guided prostate biopsies, magnetic resonance imaging , and nuclear scinitigraphy. These modalities are helpful, but have drawbacks and limitations. MRI is expensive and not mobile. Nuclear scintillation is expensive, provides low resolution planar images, and there are problems with radiotracer excretion through the kidneys. Both these modalities are not available for general use.", + "p0016": "Ultrasound imaging is relatively quick and inexpensive, and is less invasive with fewer potential side effects than other types of imaging such as X-Ray and MRI. Transrectal ultrasound guided prostate biopsy is the current standard of care for the diagnosis of prostate cancer, and may be indicated in the setting of an elevated PSA, abnormal DRE, and/or abnormality of other serum or urinary tests specific for prostate cancer .", + "p0017": "However, conventional ultrasound technology has limitations that make it unsuitable for some applications. For example, ultrasound waves do not pass well through certain types of tissues and anatomical features, and ultrasound images typically have weaker contrast and lower spatial resolution than X-Ray and MRI images. Also, ultrasonic imaging has difficulties distinguishing between acoustically homogenous tissues .", + "p0018": "In essence, conventional ultrasound images only demonstrate the possibility of revealing dark spots , which correlate very poorly with cancer. There are no identifiable patterns in the image that can be reliably correlated with cancerous tissue. In many cases prostate cancer appears as an isoechoic lesion causing high miss rate. When it is visible , it is not possible to say with certainty if it is cancer or benign because many other non-cancer conditions such as prostate atrophy, inflammation of the prostate gland, and benign tumors may also look similar in appearance on ultrasound examination. While hypoechoic areas are often seen on conventional transrectal ultrasound their value in predicting biopsy malignancy is poor, providing a positive predictive value of only 18-42% . More importantly, taking targeted biopsies of only these areas has a high false negative rate, missing clinically significant cancer in up to 48% of patients with proven disease on radical prostatectomy .", + "p0019": "A biopsy has to be performed on the suspect lesion for definitive diagnosis. Biopsies are uncomfortable and bleeding may result as a complication. Because of poor lesion detection, even the current prostate biopsy techniques miss approximately 30% of prostate cancer. Utility of color flow and power Doppler in conjunction with gray scale ultrasound has been explored, but has not achieved wide-spread use.", + "p0020": "This has resulted in ultrasound being used primarily for biopsy guidance through visualizing the prostate borders and midline in order to appropriately space and extract 8-12 systematic biopsy samples. These samples are typically taken systematically with a bias toward the lateral portions of the prostate in order to preferentially sample the peripheral zone. While they provide some level of consistency and are very likely to find larger tumors, smaller cancers and atypically located cancers may be missed. Indeed cancer detection rates using conventional systematic TRUS-biopsy range between approximately 20-50%. , meaning that some 60-80% of men undergo this procedure unnecessarily. These patients often undergo repeat or saturation biopsies, often in combination with more costly and specialized imaging such as MRI. These further procedures come with increased costs, complexity, and an increased risk of morbidity both due to the repeated procedures themselves and to potential delays in diagnosis which can lead to decreased opportunity for cure. Therefore, there is an urgent need for a new imaging methodology that will have widespread utility as a tool for primary screening and diagnosis of prostate cancer.", + "p0021": "Advances in ultrasound technology have created new high frequency imaging systems. High frequency ultrasound refers to ultrasound systems which can transmit ultrasound at a center transducer frequency of 15 MHz or higher, compared to 6-9 MHz center on conventional clinical prostate ultrasound imaging systems. Exemplary micro-ultrasound devices operate at 29 MHz . This increased frequency provides superior spatial resolution down to 70 microns, leading to 300% improved resolution over existing platforms. These improvements in resolution provide as yet unseen detail in prostate ultrasound images, which may enable improved visualization and targeting of suspicious regions as well as systematic image-guided prostate biopsy.", + "p0022": "Yet, a need remains for identifying what prostate cancer looks like in a high resolution ultrasound image of the prostate with a selected statistical confidence. Additionally there is a need for a system assisting in diagnosis of prostate cancer by reducing the rate of false negative by guiding biopsies to suspicious patches of tissue or to allow targeted biopsies of high risk tissues in the prostate.", + "p0023": "The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also be inventions.", + "p0024": "FIGURES", + "p0025": "The foregoing and other features and advantages of the subject matter disclosed herein will be made apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:", + "p0026": "FIG. 1 is a flow diagram which illustrates one embodiment of the present method for generating indicator features in accordance with the teachings hereof;", + "p0027": "FIG. 2 is a continuation of the flow diagram of FIG. 1 with flow processing continuing with respect to node A;", + "p0028": "FIG. 3 is a flow diagram which illustrates a method for using Indicator Features to train a classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0029": "FIG. 4 is a flow diagram which illustrates a method for using Indicator Features to determine relative risk and a confidence interval and optionally generate a multiclass classifier in accordance with an embodiment of the invention;", + "p0030": "FIG. 5 is a flow diagram which illustrates a method for using Indicator Features to train a binary classification tree classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0031": "FIG. 6 is a flow diagram which illustrates a method for using Indicator Features to train a random forests classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0032": "FIG. 7 is a flow diagram which illustrates a method for using Indicator Features to train a binary classification tree classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0033": "FIG. 8 is a flow diagram which illustrates a method for using Indicator Features to train an artificial neural network classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0034": "FIG. 9 is a flow diagram which illustrates a method for using Indicator Features to train a na\u00efve Bayesian network classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0035": "FIG. 10 is a flow diagram which illustrates a method for using Indicator Features to train a linear discriminant analysis classifier and optionally a multiclass classifier in accordance with an embodiment of the invention;", + "p0036": "FIG. 11 provides high resolution micro-ultrasound images demonstrating some Indicator Features obtained in accordance with an embodiment of the invention;", + "p0037": "FIG. 12 demonstrates an embodiment of a multiclass classifier of Indicator Features in high resolution micro-ultrasound images of prostate generated in accordance with an embodiment of the invention;", + "p0038": "FIG. 13 provides high resolution micro-ultrasound images of prostate illustrating needle biopsy paths in accordance with an embodiment of the invention;", + "p0039": "FIG. 14 is a block diagram of an exemplary computer system for implementing a method of generating Indicator Features in high resolution micro-ultrasound images of prostate and using them to generate a classifier and/or a multiclass classifier, in addition to use of the classifier to classify features in patient high resolution micro-ultrasound images of prostate according to an embodiment of this invention;", + "p0040": "FIG. 15 provides an exemplary Possible Features Table illustrating a list of features and feature combinations along with number of cases where the feature was assigned and when the associated biopsy sample contained cancer. This data is used to determine the \u201cUse for Training\u201d column which states \u201cNot Enough Data\u201d for cases where only 1 example is available, and Unclear/Chance level if the ratio of N /N is sufficiently close to \u00bd;", + "p0041": "FIG. 16 is a flow diagram which illustrates one embodiment of the present method for assessing patient risk for prostate cancer in accordance with the teachings hereof;", + "p0042": "FIG. 17 is a continuation of the flow diagram of FIG. 16 with flow processing continuing with respect to node B;", + "p0043": "FIG. 18 is a block diagram of one example system for performing various aspects of the present method as described with respect to the flow diagrams of FIG. 16 and FIG. 17 .", + "p0044": "DETAILED DESCRIPTION", + "p0045": "This invention provides a method of generating Indicator Features, Indicator Features and their use in a system comprising high resolution micro-ultrasound to determine a degree of risk for cancer in prostate tissue. Indicator Features are provided in high resolution micro-ultrasound images and are used to train a classifier, which can be used to determine a risk of cancer in other images, or can be grouped into a multiclass classifier, which can be used to determine a risk of cancer.", + "p0046": "A Method of Generating Indicator Features", + "p0047": "With conventional ultrasound, there was essentially only one feature available, a dark spot , which was used in the images as a possible indicator of cancer. As described in the Background, this feature correlated poorly with cancerous tissue. Thus, there was little confidence that ultrasound technology in general would be useful for identifying risk of cancer. Initially, once the prostate was able to be visualized with high resolution ultrasound, it was thought that the increased sensitivity would make the dark spot feature more reliable. Applying the traditional methodologies would have indicated that higher resolution imaging would reveal smaller dark spots, and this would be the feature to use in determining risk of cancer. This didn't work, however, indicating that the traditional way of looking for cancer using ultrasound did not work for the new high resolution systems.", + "p0048": "With the advent of high resolution micro-ultrasound, significantly more detail, which is textural in nature in B-mode images was made available providing many possible features in the images. What was not known, however, was whether a subset of these newly identifiable patterns or features in these new high resolution images could be identified, characterized and tested to become reliable indicators of risk that cancer was absent or present in the tissue being imaged, especially given the failure of conventional ultrasound to do so. Moreover, given the high incidence of issues that commonly develop in an aging prostate, such as benign prostatic hyperplasia, which also exhibits abnormal but benign multiplication of the cells, or prostatitis, it was unknown whether these other disorders would exhibit similar features when imaged at a higher resolution that would only interfere with the ability to determine features reliably indicative of cancer.", + "p0049": "What was needed by clinicians was a subset of features identifiable in high resolution ultrasound images that could be identified by clinicians and correlate with statistical significance and confidence to cancerous tissue. The number of features in this subset would need to be sufficiently large to be comprehensive, but not so large that a clinician would not be able to easily work with the list of possible indicator features when viewing an ultrasound image and making decisions while conducting a procedure of either examining a prostate for a risk of cancer or while conducting a biopsy to guide them to the tissue most likely at risk for cancer. Thus, what was needed in the art was a set of Indicator Features, which could be used by clinicians working with high resolution ultrasound systems that are indicative of benign tissue or tissue that has a very low risk of being cancerous.", + "p0050": "High resolution micro-ultrasound image is an image which is generated using a high resolution micro-ultrasound transducer to obtain electrical signals, which are then converted into an image. This image may be constituted in greyscale pixels , or may be rendered in color using known processing techniques such as Color Flow Imaging , Elastography, Contrast Enhancement, and/or Spectral Analysis. The scanning modality can be transrectal ultrasound , transperineal, transabdominal, and/or intra surgery.", + "p0051": "Overview of the Method for Generating Indicator Features", + "p0052": "General Features would be any discernable, recurring patterns, or features, appearing in high-resolution micro-ultrasound images.", + "p0053": "Possible Features are General Features, which are observed to correlate to tissue for which pathology is known, ranging from benign to the highest grade of cancer. Possible Features can be identified by using some form of Object-Based Image Analysis , and are then characterized and given a useful label.", + "p0054": "Candidate Features and/or combinations of Candidate Features will then be selected from the Possible Features and used to identify Possible Features and/or combinations thereof in a new set of training images for which the pathology is known but not provided to the reader of the image at the time of feature identification . Candidate Features differ from Possible Features in that Possible Features could have a subjective bias as they were identified in the image while knowing the pathology of the corresponding tissue. Candidate Features are Possible Features which have been identified in tissue while the reader of the image was blinded to the pathology.", + "p0055": "Candidate Features, which have been identified in a set of training images, will be used to populate a Candidate Feature Table and will be assigned the pathology score. Additional patient data may be included in the Candidate Feature Table to generate an Augmented Feature Table. The Feature Table or Augmented Feature Table will be used to train a linear or non-linear classifier to determine the predictive probability of a Candidate Feature and/or combinations of Candidate Features being indicative of benign tissue or a grade of cancerous tissue. Candidate Feature and/or combinations of Candidate Features, which demonstrate little or no statistical significance, will be eliminated to yield Indicator Features", + "p0056": "Indicator Features are features appearing alone and/or in combination with other Indicator Features in high resolution micro-ultrasound images, which have been determined to be significantly statistically correlated to either benign tissue or some grade of cancerous tissue on the basis of predictive probabilities.", + "p0057": "Reference is now being made to the flow diagram of FIG. 1 , which illustrates one embodiment of the present method for generating Indicator Features in accordance with the teachings hereof. Flow processing begins at step 100 and immediately proceeds to step 102 . At step 102 , receive a set of high resolution micro-ultrasound images of prostate tissue, which corresponds to tissue for which the pathology has been determined. At step 104 , analyze the image to detect Possible Features by, for example, conducting some form of object-based image analysis in the areas of the image where the pathology of the tissue in the image is known. At step 106 , characterize and provide a unique label to all detectable Possible Features. At step 108 , add Possible Features thereby detected and/or combinations of Possible Features observed to occur together into the List of Possible Features. At step 110 , repeat steps 104 through 108 until all Possible Features in the set have been identified to produce the Set of Possible Features at step 112 .", + "p0058": "Reference is now being made to the flow diagram of FIG. 2 , which is a continuation of the flow diagram of FIG. 1 , continuing with respect to node A. At step 122 , obtain a set of high resolution micro-ultrasound images of prostate tissue that corresponds to tissue for which the pathology is known. It is possible to use the same training set but preferred to have a new set. At step 124 , a reader identifies Possible Features present in the images corresponding to the area for which the pathology of the tissue is known, while blinded to the pathology to generate Candidate Features. At step 126 , Enter Candidate Features into a Candidate Feature Table and correlate the pathology to generate a Candidate Feature Table, optionally additional patient data can be added to generate an Augmented Candidate Feature Table. At step 128 , Use the Candidate Feature Table or Augmented Candidate Feature Table to train a linear or non-linear classifier. At step 130 , eliminate Candidate Features and/or combinations of Candidate Features which are not statistically significant to produce a plurality of Indicator Features at step 132 . If sufficient Indicator Features have been generated, then processing stops.", + "p0059": "It should be appreciated that the flow diagrams depicted herein are illustrative. One or more of the operations may be performed in a different manner. Other operations may be added, modified, enhanced or consolidated. Variations thereof are intended to fall within the scope of the appended claims.", + "p0060": "Indicator Features can then be used to generate an Indicator Feature Table, which can be used to train a classifier, which can then be used to classify future patient images. Additional patient data may be included in the Indicator Feature Table to generate an Augmented Indicator Feature Table. There are a number of algorithms known to one skilled in the art that can be used to analyze the Indicator Feature Table, which constitutes a training set to generate a predictive probability for the correlation between the presence of a feature in an image and the presence of cancer in the tissue being imaged. The classifier can be used to classify the features in images of patients generated using high-resolution micro-ultrasound devices.", + "p0061": "Alternatively, these predicted probabilities can be used to create a multiclass classifier by grouping and ranking the features according to the similarities and differences between their weights or predicted probabilities. In this embodiment, the multiclass classifier can be used to classify the features in images of patients generated using high-resolution micro-ultrasound devices.", + "p0062": "Reference is now being made to the flow diagram of FIG. 3 which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. Flow processing begins at step 136 and immediately proceeds to step 138 . At step 138 , provide a set of Indicator Features. At step 140 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 142 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 144 , the Indicator Feature Table or the Augmented Feature Table is used to train a linear or non-linear classifier. The process can stop here, or optionally, proceed to step 162 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 148 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. The classifier or the multiclass classifier can be further trained using either the training data or test data.", + "p0063": "High Resolution Micro-Ultrasound Images", + "p0064": "The desired ultrasound for use with the disclosed methods can be applied, transmitted and received using an ultrasonic scanning device that can supply ultrasound at a center frequency sufficient to accurately resolve sonographic features on the order of 70 um. For example, a system with a center frequency transmit of at least about 15 MHz to the highest practical frequency can be used. In exemplary aspects, ultrasound can be supplied at a center frequency of 15 MHz, 16 MHz, 17 MHz, 18 MHz, 19 MHz, 20 MHz, 21 MHz, 22 MHz, 23 MHz, 24 MHz, 25 MHz, 26 MHz, 30 MHz, 31 MHz, 32 MHz, 35 MHz, 40 MHz, 45 MHz, 50 MHz, 55 MHz, 60 MHz, 65 MHz, 70 MHz, or higher than 70 MHz. Thus, an ultrasound system or device capable of operating at a center frequency at 15 MHz or above can be used.", + "p0065": "One such exemplary system is the 29 MHz transrectal micro-ultrasound system and transducer . Another such exemplary system can have the components and functionality described in U.S. Provisional Application No. 62/253,310, which is incorporated herein by reference. Another exemplary system can have components and functionality described in PCT/IB2015/053458, which is incorporated herein by reference. Another such exemplary system can have components and functionality described in PCT/IB2015/052882, which is incorporated herein by reference.", + "p0066": "It is contemplated that any system capable of producing an ultrasound image using a high frequency micro-ultrasound can be used. Thus, the methods can be practiced using a mechanically scanned ultrasound system that can translate an ultrasound beam as it sweeps along a path. The methods can also be practiced using an array based system where the beam is translated by electrical steering of an ultrasound beam along the elements of the transducer. One skilled in the art will readily appreciate that beams translated from either type system can be used in the described methods, without any limitation to the type of system employed. The type of system is therefore not intended to be a limitation to any described method because array and mechanically scanned systems can be used interchangeably to perform the described methods.", + "p0067": "Needle Biopsy During Ultrasound Imaging", + "p0068": "Embodiments described herein include a side-fire ultrasonic probe with an alignment feature that, when used to connect the probe to a needle guide for intra-cavity medical procedures, facilitates alignment of one or more needles translated through the needle guide with an imaging plane of an ultrasonic transducer. The alignment feature is configured such that alignment of a needle within the imaging plane is accomplished even when a protective sheath is disposed between the alignment feature and the needle guide.", + "p0069": "By positioning the translated needles within the imaging plane of a side-fire type ultrasonic probe, an ultrasonic image can be used to image an advancing needle with respect to an intra-cavity structure of interest. This ability is particularly useful when the ultrasonic transducer has a frequency and/or resolution sufficient to image intra-structure or intra-organ features. Simultaneously imaging the structure of interest and the needle permits navigation of the needle to a specific intra-cavity structure within a human body, or, given sufficient resolution of the ultrasonic transducer, navigation of the needle to a specific location within the structure. This can then improve the diagnostic capability of the procedure or effectiveness of the therapy. Allowing for positioning of a needle oriented at different angles with respect to the probe enables access to a range of locations within the body or structure by the needles while reducing the manipulation of the probe. This can improve patient comfort during the procedure, as well as patient safety.", + "p0070": "In an embodiment, an ultrasonic probe includes a cylindrical housing that includes a needle guide alignment feature on the surface of the housing. The alignment feature is used to connect a needle guide to the cylindrical housing and to align the needle guide such that a needle translated through the guide is translated in an imaging plane of the ultrasonic transducer. The alignment feature is configured such that the needle is aligned in the imaging plane even when a protective sheath is disposed between the housing and the needle guide. An exemplary needle guide alignment feature is provided by U.S. Pat. No. 9,113,825, incorporated by reference herein.", + "p0071": "Generating a List of Possible Indicator Features", + "p0072": "Possible Features are discemable, recurring patterns, or features, appearing in high-resolution micro-ultrasound images, which are observed to correlate to tissue for which pathology is known, ranging from benign to the highest grade of cancer. For example, the Gleason Sum scale ranges from Benign, 6, 7, 8, 9, & 10. Note that 2-5 used to be used as well but are generally now just labelled as benign. Other scales for grading the degree of cancer, or the stage of the cancer, in the tissue, though the concept remains the same. Possible Features are qualitative, not quantitative.", + "p0073": "Possible Features can be identified by using some form of Object-Based Image Analysis , which employs two main processes, segmentation and classification. Possible Features are then characterized and given a useful label. A label could be simply Feature A, Feature B, Feature C, etc. A label could be given a descriptive name that describes the texture present in the image that is somewhat intuitive such a \u201cbright spot,\u201d or \u201cmottled.\u201d This process is conducted for all the Possible Features that can be identified along the needle path of the biopsy needle or in a region corresponding to biopsied tissue.", + "p0074": "Traditional image segmentation is on a per-pixel basis. However, OBIA groups pixels into homogeneous objects. These objects can have different shapes and scale. Objects also have statistics associated with them which can be used to classify objects. Statistics can include geometry, context and texture of image objects. Various systems have been developed for detecting and analyzing target patterns in digital images. A digital image is a rectangular array of pixels. Each pixel is characterized by its position in the array and a plurality of numerical pixel values associated with the pixel. The pixel values represent color information for various image layers. For example, grayscale digital images are represented by a single image layer, whereas RGB true-color images are represented by three image layers. Some existing analysis systems apply semantic networks to analyze the contents of the digital images. Systems that apply semantic networks perform object-oriented picture analysis, as opposed to solely statistical pixel-oriented analysis. Consequently, semantic network systems classify not only pixels, but also data objects linked to the pixels. The data objects that are linked to the pixels and to one another represent measurable information about the digital images.", + "p0075": "Possible Features are identified by image segmentation in the region of the image corresponding to tissue, which has been biopsied. It is the process of determining areas with similar image characteristics and which contrast to adjacent regions that are discernably different with respect to the same characteristic. For example, if the image is a greyscale image, an area , which are almost white in color and appear adjacent to and surrounded by other areas, which are dark grey in color with a sharp contrast boundary between the two, the white area might be labelled a \u201cbright spot.\u201d In another greyscale example, an area might appear light grey with \u201cfuzzy\u201d boundaries of slightly darker area, might be grouped together and labelled \u201cmottled.\u201d Variations between areas may occur on a distinctive spatial scale , and may be described by relative changes in average brightness, sharpness and regularity of borders, and/or changes in gray-level textural parameters such as entropy, variability, contrast, or correlation. Regions with parameters different from typical normal tissue may be segmented and grouped based on these parameters.", + "p0076": "In an embodiment, a human is used to generate the list of Potential Features. In an embodiment, a computer can be used to read the images and identify Possible Features and/or combinations of Possible Features in the images. In an embodiment, a human assisted by a computer can read the images and identify Possible Features and/or combinations of Possible Features in the images.", + "p0077": "Exemplary Potential Features", + "p0078": "In an embodiment, the first step entails determining distinguishable recurring patterns in high resolution transrectal micro-ultrasound images of prostate glands. In an embodiment, a 29 MHz transrectal micro-ultrasound system and transducer is used to acquire cine loops during each biopsy taken using the high resolution micro-ultrasound platform on patients who have had biopsies or radical prostatectomy such that clinical analysis data exists to determine whether the image correlates with cancerous tissue or not. The act of acquiring cine loops would be known to one skilled in the art as involving preparation of the transducer, insertion of the transducer into the rectum, adjustment of imaging settings to optimize image quality and through interrogation of the prostate gland to identify correct biopsy locations. These biopsy locations may be selected based on anatomy or based on the sonographic features noted in the subject .", + "p0079": "In an embodiment, whole-mount radical prostatectomy slides registered to micro-ultrasound images can be used as a starting point to suggest useful imaging features.", + "p0080": "In an embodiment randomly-selected micro-ultrasound cine loops, each captured during one specific biopsy can be analyzed by one or more experts to determine imaging findings along the biopsy needle path indicative of carcinomas, as well as normal tissue.", + "p0081": "In these embodiments at this stage of the process, the radical prostatectomy slides or biopsy data are not used to assign a link between a feature and cancerous tissue, but to assist in noticing which distinguishable and recurring aspects of the images might be suggestive or consistently present in tissue having a certain stage of prostate cancer.", + "p0082": "In order to generate a list of ultrasound image features, a series of images or video cine loops can be reviewed by a series of experts to identify a large set of distinguishable, recurring features. Each reviewer assigns a term to any non-normal or non-uniform pattern encountered, then the reviewers meet to compare terms and select the most common nomenclature for each feature. In an embodiment, text descriptions may be analyzed by a computer using natural language processing systems to extract these features.", + "p0083": "In an embodiment, a series of still images and/or cine loops can be analyzed by a computer for pattern detection to determine a list of distinguishable and recurring features existing in the images, presenting potential candidate features which might be found to be consistently able to be correlated to a stage or risk of prostate cancer.", + "p0084": "In an embodiment, the Possible Features comprise features having the same or similar characteristics as:", + "p0085": "Feature Heterogenous-Bright Echoes-Finger/Funky Shadows Heterogenous-Finger/Funky Shadows Heterogenous-Coarse-Shadows Heterogenous-Hyperechoic-Coarse Heterogenous-Irregular Heterogenous-Lesion Heterogenous-Coarse-Finger/Funky Shadows Heterogenous-Irregular-Coarse Heterogenous-Shadows Heterogenous-Defined Heterogenous-Defined-Coarse Heterogenous-Hyperechoic Heterogenous-Hyperechoic-Irregular Heterogenous-Hypoechoic-Lesion-Bright Echoes Heterogenous \u201cCauliflower\u201d Heterogenous \u201cSmudged\u201d Heterogenous \u201cMottled\u201d Heterogenous \u201cCauliflower/Smudgy/Mottled\u201d Mild Heterogeneity Bright Echoes Bright Echoes in Hyperechoic Tissue Mixed Echo Legions Hyperechoic-Coarse-Shadows Hyperechoic-Hypoechoic-Defined Hyperechoic-Hypoechoic-Lesion-Bright Echoes Hyperechoic With or Without Ductal Patches Hypoechoic Hypoechoic-Bright Echoes-Coarse Heterogenous-Bright Echoes-Coarse Heterogenous-Bright Echoes Heterogenous-Coarse Hyperechoic-Hypoechoic-Lesion-Coarse Hypoechoic-Defined Hyperechoic-Bright Echoes-Coarse Anechoic-Heterogenous-Defined Heterogenous Heterogenous-Hypoechoic-Lesion-Coarse Hyperechoic-Coarse-Irregular Prostate Borders Hyperechoic-Coarse-Microgranulomas Hyperechoic-Hypoechoic-Bright Echoes Hyperechoic-Hypoechoic-Lesion Hypoechoic-Lesion-Bright Echoes-Coarse Hypoechoic-Undefined/Vague Hyperechoic-Hypoechoic Hyperechoic-Bright Echoes Hyperechoic-Coarse Anechoic-Hyperechoic-Defined Hyperechoic Irregular Prostate Irregular Prostate Boarder Bright Echoes Bright Echoes-Coarse Coarse Heterogenous-Hyperechoic-Lesion Coarse-Small Ducts Small Ducts Small regular ducts \u201cSwiss cheese\u201d", + "p0086": "Generating a Set of Candidate Features", + "p0087": "According to an embodiment, the Possible Features are then further processed by having someone skilled in the art of reading images, a computer or a combination thereof, identify the presence or absence of a Possible Features in a set of new training images, but blinded to the pathology while recording the presence or absence of features along the path of the biopsy needle in the image. Only features that overlap with the biopsy path are used. Any of the Possible Features that are assigned to images in the new training set are now considered Candidate Features. Since the individual is blinded to the pathology of the biopsy results, this step accounts for possible subjective interpretation biases of the features by the individual. If combinations of features are observed to occur together, then combinations can be included and treated as a singular feature, ie., when the individual features are present alone, then they are treated as one piece of data, but when the features are present together, they are treated both as one piece of data and also recorded as individual features.", + "p0088": "Then the results are used to construct a Candidate Feature Table, where they are correlated to the actual pathology of the tissue imaged in the region where the Candidate Feature was identified.", + "p0089": "In an embodiment, similar steps are followed to identify the images, but whole-mount radical prostatectomy slides registered to micro-ultrasound images can be used to further process the Set of Candidate Indicator Features.", + "p0090": "In an embodiment, a computer can be used to read the images and identify Candidate Features and/or combinations of Candidate Features in the new training set of images.", + "p0091": "In another embodiment, other patient data is analyzed along with the features , to increase the confidence that a specific feature is associated with cancer, comprising data obtained from: spatial information about the image, spatial information about the features, 3-D information, spectral information about the image, and, images collected using other modalities such as B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarkers, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies", + "p0092": "This additional information is then included in the Augmented Candidate Feature Table along with each feature or combination of features.", + "p0093": "Feature N N A B A + B C D A + D Mean Value Augmented Patient Data Mean Value PSA Doppler Value Elastography Value", + "p0094": "Candidate Feature and/or combinations of Candidate Features, which demonstrate little or no statistical significance will be eliminated to yield Indicator Features. One skilled in the art would appreciate that the range of the confidence interval on predictive probability spanning 1 constitutes the threshold of statistical significance.", + "p0095": "An Exemplary Set of Candidate Features", + "p0096": "Using the Possible Features, Candidate Features can be identified in new set of training images, for which the pathology is known, but blinded to the reader of the images.", + "p0097": "In an embodiment, a set of Possible Features to be used to identify Candidate Features comprise features having the same or similar characteristics as", + "p0098": "- a) small regular ducts \u201cSwiss cheese\u201d - b) hyperechoic with or without ductal patches - c) mild heterogeneity - d) bright echoes in hyperechoic tissue - e) heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance - f) bright echoes - g) irregular prostate - h) irregular prostate border - i) mixed-echo lesions - j) irregular shadowing", + "p0099": "In an embodiment, a set of Possible Features to be used to identify Candidate Features comprise features having the same or similar characteristics as:", + "p0100": "- a) regular ductal pattern \u201cSwiss cheese\u201d; - b) hyperechoic with ductal patches; - c) mild heterogeneity with small bright echoes in hyperechoic tissue; - d) bright echoes; - e) heterogeneous \u201csmudged/mottled\u201d; - f) heterogeneous \u201ccauliflower\u201d; - g) irregular shadowing; and - h) mixed echo lesion causing irregular prostate borders.", + "p0101": "For example, in an embodiment 200 randomly-selected micro-ultrasound cine loops, each captured during one specific biopsy , can be analysed by one or more experts to determine imaging findings along the biopsy needle path indicative of high grade and low-intermediate grade carcinomas, as well as normal tissue. For example, the sample set can include 100 benign, 50 low-grade and 50 high-grade biopsy-proven samples. In an embodiment, experts can be blinded to the pathological findings of each biopsy sample, and record the variety of Potential Features present along the needle path in each image, which once identified become Candidate Features. These Candidate Features can then be analyzed to see how often they are associated with biopsy-proven benign tissue or cancer.", + "p0102": "In another embodiment, other information is analyzed along with the features , to increase the confidence that a specific feature is associated with cancer, comprising, with additional data pertaining to the patient based on data obtained from: B-mode images, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies.", + "p0103": "TABLE 1 An exemplary Candidate Feature Table comprising a list of sonographic features for which biopsy results are attributed. N N Feature Small regular ducts \u201cSwiss cheese\u201d 7 1 Hyperechoic, with or without ductal patches 50 14 Mild heterogeneity 42 24 Bright Echoes in hyperechoic tissue 10 4 Heterogeneous \u201ccauliflower/smudgy/mottled\u201d 32 22 appearance Bright Echoes 30 18 Irregular Prostate 1 1 Irregular PZ border 1 1 Mixed-echo lesions 2 2 Irregular Shadowing 12 11", + "p0104": "Determining a Risk Parameter for Each Candidate Feature or Combination of Candidate Features", + "p0105": "There are various statistical univariate and multivariate methodologies that can be used to assign a Risk Parameter to each sonographic feature presented in the Pathology Set of Pathology Training Features. The objective is to determine when a feature is present in a sonographic image, what is the probability that it is associated with the tissue being imaged having a grade of prostate cancer versus benign tissue.", + "p0106": "Expressed in terms of predictive analytics, the pathological diagnosis is the dependent variable, the sonographic feature is the independent variable and the mathematical task is to ascertain the likelihood that the presence of the independent variable is dependent upon the presence of cancer in that tissue. The answer can be binomial in that the equation seeks to prove either cancer is present or absent with each feature. Alternatively, the answer may be expressed as a beta random variable, expressing the increased or decreased risk of cancer on a continuous scale between 0 and 1 . This continuous scale may also be expressed as an odds ratio or relative risk between 0 and infinity .", + "p0107": "In an embodiment, a univariate relative risk and a confidence interval are calculated for each sonographic feature, and used to assign a risk parameter to each Candidate Feature and/or combination of Candidate Features. Candidate Features and/or combinations of Candidate Features which do not demonstrate sufficient statistical significance will be eliminated to generate the Indicator Features.", + "p0108": "Indicator Features", + "p0109": "Indicator Features are features appearing alone and/or in combination with other Indicator Features in high resolution micro-ultrasound images, which have been determined to be significantly statistically correlated to either benign tissue or some grade of cancerous tissue on the basis of predictive probabilities.", + "p0110": "Indicator Features can then be used to generate an Indicator Feature Table, which can be used to train a classifier, which can then be used to classify future patient images. There are a number of algorithms known to one skilled in the art that can be used to analyze the Indicator Feature Table, which constitutes a training set to generate a predictive probability for the correlation between the presence of a feature in an image and the presence of cancer in the tissue being imaged. The classifier can be used to classify the features in images of patients generated using high-resolution micro-ultrasound devices.", + "p0111": "Alternatively, these predicted probabilities can be used to create a multiclass classifier by grouping and ranking the features according to the similarities and differences between their weights or predicted probabilities. In this embodiment, the multiclass classifier can be used to classify the features in images of patients generated using high-resolution micro-ultrasound devices.", + "p0112": "Exemplary Indicator Features", + "p0113": "In an embodiment, the Indicator Features comprise features having the same or similar characteristics as", + "p0114": "small regular ducts \u201cSwiss cheese;\u201d", + "p0115": "regular ductal pattern \u201cSwiss cheese\u201d;", + "p0116": "hyperechoic with ductal patches;", + "p0117": "hyperechoic with or without ductal patches;", + "p0118": "mild heterogeneity with small bright echoes in hyperechoic tissue;", + "p0119": "mild heterogeneity;", + "p0120": "bright echoes in hyperechoic tissue;", + "p0121": "heterogeneous \u201csmudged/mottled\u201d;", + "p0122": "heterogeneous \u201ccauliflower\u201d;", + "p0123": "heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance;", + "p0124": "bright echoes;", + "p0125": "irregular prostate ;", + "p0126": "irregular prostate border;", + "p0127": "mixed-echo lesions;", + "p0128": "irregular shadowing; or", + "p0129": "mixed echo lesion causing irregular prostate borders.", + "p0130": "A set of Indicator Features can then be used to generate an Indicator Feature Table. In an embodiment, this is generated by re-using the data that used to evaluate the Candidate Features . The Table can be used to train a classifier, which can then be used to classify future patient images. The results of this classifier can optionally be grouped to generate a multiclass classifier, which then can be used to classify future patient images.", + "p0131": "In an embodiment, a new set of images can be used to identify Indicator Features in the images, while blinded to the pathology, then use the results of assigning Indicator Features to the new set of images plus the corresponding pathology to populate an Indicator Feature Table. Optionally, additional patient data can be included in the Table. The Table can be used to train a classifier, which can then be used to classify future patient images. The results of this classifier can optionally be grouped to generate a multiclass classifier, which then can be used to classify future patient images.", + "p0132": "In an embodiment, a set of Indicator Features comprise features having the same or similar characteristics as:", + "p0133": "- a) small regular ducts \u201cSwiss cheese\u201d - b) hyperechoic with or without ductal patches - c) mild heterogeneity - d) bright echoes in hyperechoic tissue - e) heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance - f) bright echoes - g) irregular prostate - h) irregular prostate border - i) mixed-echo lesions - j) irregular shadowing", + "p0134": "In an embodiment, a set of Indicator Features comprise features having the same or similar characteristics as:", + "p0135": "- a) regular ductal pattern \u201cSwiss cheese\u201d; - b) hyperechoic with ductal patches; - c) mild heterogeneity with small bright echoes in hyperechoic tissue; - d) bright echoes; - e) heterogeneous \u201csmudged/mottled\u201d; - f) heterogeneous \u201ccauliflower\u201d; - g) irregular shadowing; and - h) mixed echo lesion causing irregular prostate borders.", + "p0136": "In another embodiment, other patient data is analyzed along with the features , to increase the confidence that a specific feature is associated with cancer, comprising data obtained from, B-mode images, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies.", + "p0137": "This additional information is then included in an Augmented Indicator Feature Table along with each feature or combination of features.", + "p0138": "Feature N N A B A + B C D A + D Mean Value Augmented Patient Data Mean Value PSA Doppler Value Elastography Value", + "p0139": "In an embodiment, a training set comprising Indicator Features, which have been correlated to clinical pathology data for the same tissue imaged in the sonogram. In an embodiment, this is accomplished by using a high-resolution transrectal micro-ultrasound system to acquire images of the needle path followed during each biopsy taken on patients. The pathological analysis of the tissue collected in each biopsy is then used determine image findings along the biopsy needle path indicative of carcinomas, as well as normal tissue.", + "p0140": "In an embodiment, a univariate relative risk and a confidence interval are calculated for each sonographic feature, and used to assign a risk parameter to each Indicator Feature and/or combination of Indicator Features.", + "p0141": "In another embodiment, the calculating step uses machine learning. Common machine learning algorithms may be used to create a classifier based on the feature data, such as binary classification trees, \u201crandom forests\u201d, support vector machines, artificial neural networks, and na\u00efve bayesian networks. In another embodiment, multivariate relative risk is calculated for a combination of features. A skilled person would understand as the number of features increases machine learning is used since the complexity of selecting appropriate thresholds and managing generalizability increases exponentially with the number of features or combination of features", + "p0142": "Calculating a Univariate RR and CI for Each Indicator Feature and/or Combination of Indicator Features", + "p0143": "FIG. 4 illustrates the process of building a multiclass classifier calculating a univariate relative risk and a confidence interval for each sonographic feature, and used to assign a risk parameter to each sonographic feature in the Indicator Features Table, which can be then used to generate a multiclass classifier. The RR ratio is the relative occurrence of a given feature or symptom in two distinct populations. In other words, the RR for each sonographic feature indicates the relative occurrence of each feature in a population of cancerous tissue versus non-cancerous tissue. Large differences in occurrence rates suggest that the feature may be used to predict which population a given image or patient belongs to. Equation is used to calculate a relative risk ratio of each Indicator Feature.", + "p0144": "TABLE 2 Example variables for Relative Risk Ratio Calculation Disease Disease Present Absent Feature A B Present Feature C D Absent", + "p0145": "The statistical significance of this ratio may be approximated by assuming a log-normal distribution with mean ln and variance given in Equation 2.", + "p0147": "So that the 95% confidence interval may be calculated by:", + "p0148": "CI= e m\u00b11.96\u03c3", + "p0149": "Table 3 illustrates relative risks and confidence intervals calculated for example Indicator Features from blinded analysis of 100 biopsy-proven benign and 100 biopsy-proven malignant cine loops", + "p0150": "N N Indicator Features RR Small regular ducts \u201cSwiss 7 1 0.28 cheese\u201d Hyperechoic, with or without 50 14 0.49 ductal patches Mild heterogeneity 42 24 1.19 Bright Echoes in hyperechoic 10 4 0.79 tissue Heterogeneous \u201ccauliflower/ 32 22 1.48 smudgy/mottled\u201d appearance Bright Echoes 30 18 1.24 Irregular Prostate 1 1 2.01 Irregular PZ border 1 1 2.01 Mixed-echo lesions 2 2 2.02 Irregular Shadowing 12 11 1.94 Note that in this example, no correction was made to account for the number of features tested. Such a correction is warranted and would likely widen the 95% confidence intervals, but should not affect the ranking of features.", + "p0151": "Building a Multiclass Classifier", + "p0152": "Reference is now being made to the flow diagram of FIG. 4 which illustrates one embodiment of the present method for generating a classifier comprising high resolution ultrasound images. Flow processing begins at step 152 and immediately proceeds to step 154 . At step 154 , provide a set of Indicator Features. At step 156 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 158 , the pathology data is linked to the sonographic features because the tissue sample and training image are taken from the same physical location in the prostate. At step 160 , a univariate relative risk and a confidence interval are calculated for each sonographic feature or group of features. At step 162 , group the features based on the relative risk and the confidence interval of each individual feature. At step 164 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals.", + "p0153": "Grouping Features to Build a Multiclass Classifier", + "p0154": "In an embodiment using a relative risk and confidence interval classifier, the singular features are grouped based on the relative risk and confidence interval of each individual feature. If a linear classifier is used, combinations of features may also be added if sufficient data is available. A new multiclass classifier is formed based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals.", + "p0155": "In an embodiment, predefined thresholds are used to group features based on mean relative risk values. For a data set with balanced numbers of cancerous and benign observations, these predetermined values split the theoretical 0-2 range of the relative risk value into sections that provide useful assessment of the underlying risk, for example:", + "p0156": "Risk Score Mean RR Range Assignment 1 0-0.4 Very low risk 2 0.4-0.6 Some risk 3 0.6-1.2 Indeterminate risk 4 1.2-1.6 Significant risk 5 1.6+ Very High risk", + "p0157": "In an embodiment, automated thresholding is used to determine the output scale. The following procedure may be used:", + "p0158": "- 1. The pair of values with closest mean RR are chosen and their CI ranges compared. If the CI ranges overlap by a certain percentage, the values are combined. The new mean for the group is taken as the mean of the component RR values, and the CI as the intersection of the individual CI ranges. - 2. Step 1 is repeated until no further merges are possible. - 3. Steps 1 and 2 are repeated for the pair of values with the second closest mean RR values, etc. Until every pair has been checked without a merge. - 4. The resulting groups are sorted by their mean RR values and assigned Risk Scores from 1 to N . The overlap percentage may be adjusted to give a desirable number of classes for training purposes.", + "p0159": "Alternatively, k-nearest neighbors clustering may be used with a fixed number of predefined starting classes spaced evenly over the RR space.", + "p0160": "In an embodiment, the point scale may be less than 5 points. In an embodiment, the point scale may be 5 points. In an embodiment, the point scale may be 6 points. In an embodiment, the point scale may be 7 points. In an embodiment, the point scale may be 8 points. In an embodiment, the point scale may be 9 points. In an embodiment, the point scale may be 10 points. In an embodiment, the point scale may be more than 10 points.", + "p0161": "In an embodiment, the point scale may include sub classifications. For example, the point scale is 1, 2, 3A, 3B, 3C, 4, 5.", + "p0162": "In an embodiment, the machine learning analysis is used to rank and group features with similar risk profiles into an easy to use point scale.", + "p0163": "Table 4 illustrates how the RR and CI results can be used to group and assign a Risk Score to the Indicator Features, thereby creating a multiclass classifier for high resolution micro ultrasound sonograms of the prostate.", + "p0164": "N N Assigned cer) Score Small regular ducts 7 1 0.28 1 \u201cSwiss cheese\u201d Hyperechoic, with or without 50 14 0.49 2 ductal patches Mild heterogeneity 42 24 1.19 3 Bright Echoes in hyperechoic 10 4 0.79 3 tissue Mild heterogeneity + 4 Bright Echoes in hyperechoic tissue Heterogeneous \u201ccauliflower/ 32 22 1.48 4 smudgy/mottled\u201d appearance Bright Echoes 30 18 1.24 4 Irregular Prostate 1 1 2.01 5 Irregular PZ border 1 1 2.01 5 Mixed-echo lesions 2 2 2.02 5 Irregular Shadowing 12 11 1.94 5", + "p0165": "In an embodiment, the multiclass classifier comprises a possible risk level of:", + "p0166": "- 1. 1 for small regular ducts \u201cSwiss cheese\u201d with no other heterogeneity or bright echoes; - 2. 2 for hyperechoic, with or without ductal patches ; - 3. 3 for mild heterogeneity or bright echoes in hyperechoic tissue; - 4. 4 for heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance or bright echoes ; and - 5. 5 for irregular shadowing or mixed-echo lesions or irregular prostate and/or PZ border", + "p0167": "In an embodiment, the multiclass classifier comprises a risk level of:", + "p0168": "- 1. 1 for small regular ducts or \u201cSwiss Cheese\u201d feature; - 2. 2 for hyperchoic with/without ductal patches features; - 3. 3 for a mild heterogeneity or bright echoes in hyperechoic tissue; - 4. 4 for heterogeneous \u201ccauliflower, smudgy or mottled\u201d or bright echoes \u201cstarry sky\u201d features; and - 5. 5 for irregular shadowing or mixed-echo lesions or irregular prostrate/PZ border.", + "p0169": "Linear and Non-Linear Classifiers", + "p0170": "In an embodiment, RR and CI results are used to generate predictive probabilities for Indicator Features. One skilled in the art, however, would appreciate that it would therefore be possible to using an alternative mathematical processing, such as binary classification trees, \u201crandom forests\u201d, support vector machines, artificial neural networks, and na\u00efve bayesian networks, of the same results found in the Indicator Feature Table or Augmented Indicator Feature Table. In all of these cases each Indicator Feature and/or combination of Indicator Features is considered as a separate column of a feature matrix, which may take on Boolean values . One row of this matrix is created for each image/cine loop available. Pathological result is also modelled as a boolean value in the form of a vector with one entry per image/cine loop indicating the presence or absence of cancer.", + "p0171": "Binary Classification Trees", + "p0172": "Reference is now being made to the flow diagram of FIG. 5 which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 5 illustrates the process of training a binary classification tree classifier using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 172 and immediately proceeds to step 174 . At step 174 , provide a set of Indicator Features. At step 176 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 178 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 180 , the Indicator Feature Table or the Augmented Feature Table is used to train a binary classification tree classifier. The process can stop here at step 186 , or optionally, proceed to step 182 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 184 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0173": "A binary classification tree may be created using the following algorithm:", + "p0174": "For each Indicator Feature, split the dataset based on the presence or absence of the feature.", + "p0175": "- Calculate a purity measure such as deviance, or Gini's diversity - Select the feature that leads to the split with the highest purity and create two daughter nodes, one with the presence of the feature and one with the absence", + "p0176": "For each daughter node,", + "p0177": "- If the size of the daughter node is above a threshold N entries , repeat 1 and 2 iteratively on the node. - If the size of the daughter node is below the threshold N, stop.", + "p0178": "The classifier is queried by starting at the root node and following each branch based on the input until a \u201cleaf\u201d or terminal node is reached. One may use the ratio of positive cases at the terminal node as the output. For example, if a terminal node contained 3 benign and 7 cancerous cases the output would be 70%.", + "p0179": "Random Forests", + "p0180": "Reference is now being made to the flow diagram of FIG. 6 , which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 6 illustrates the process of training a random forest classifier using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 192 and immediately proceeds to step 194 . At step 194 , provide a set of Indicator Features. At step 196 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 198 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 200 , the Indicator Feature Table or the Augmented Feature Table is used to train a random forest classifier. The process can stop here at step 206 , or optionally, proceed to step 202 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 204 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0181": "A random forest classifier may be created using the following algorithm:", + "p0182": "Randomly select a subset of the data", + "p0183": "Follow the procedure for generating a Binary Classification Tree based using this subset", + "p0184": "Save the resulting tree, and repeat from step 1 N times to produce N distinct classification trees", + "p0185": "The classifier is queried by querying each individual tree with a given input, and taking the average response of the set of trees. For example, for the input case \u201cBright echoes AND irregular border\u201d the trees may range in suspicion of cancer from 70-90%, one might report the outcome as having a mean value of 80% with a confidence interval of 72-85%.", + "p0186": "Support Vector Machines", + "p0187": "Reference is now being made to the flow diagram of FIG. 7 , which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 7 illustrates the process of training a support vector machine using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 208 and immediately proceeds to step 210 . At step 210 , provide a set of Indicator Features. At step 220 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 222 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 224 , the Indicator Feature Table or the Augmented Feature Table is used to train a support vector machine. The process can stop here at step 230 , or optionally, proceed to step 226 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 228 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0188": "Quadratic programming techniques such as Sequential Minimization Optimization or Iterative Single Data Algorithm may be applied to determine the N-dimensional hyperplane best separating the benign from cancerous data points . A kernel function is not required so long as independent, binary feature values are used. Output \u201cscore\u201d or posterior probability of cancer may be determined through the linear equation f =x\u2032\u03b2+\u03b20, where beta is the vector orthogonal to the classification hyperplane.", + "p0189": "Artificial Neural Networks", + "p0190": "Reference is now being made to the flow diagram of FIG. 8 , which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 8 illustrates the process of training an artificial neural network classifier using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 232 and immediately proceeds to step 234 . At step 234 , provide a set of Indicator Features. At step 238 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 238 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 240 , the Indicator Feature Table or the Augmented Feature Table is used to train an artificial neural network classifier. The process can stop here at step 246 , or optionally, proceed to step 242 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 244 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0191": "An artificial neural network may be created with N input nodes and a single output node with a continuous output representing the probability of cancer. In between these input and output nodes are M layers of perceptrons each summing weighted input from the previous layer and exhibiting a sigmoidal response curve. These weights are determined through a backpropagation algorithm, which is well described in the literature. Briefly:", + "p0192": "- 1. Split the data into training and validation sets . - 2. For a given training example, and starting set of weights the response of each layer is calculated sequentially until the output is reached. - 3. The error between the output and truth is calculated and propagated backwards through the system layer-by-layer, modifying each weight by a portion of the gradient. - 4. Calculate error in validation set. - 5. Repeat from 2 for each training example, using the current set of weights until validation set error is no longer decreasing.", + "p0193": "Na\u00efve Bayesian Networks", + "p0194": "Reference is now being made to the flow diagram of FIG. 9 , which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 9 illustrates the process of training a na\u00efve Bayesian network classifier using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 272 and immediately proceeds to step 274 . At step 274 , provide a set of Indicator Features. At step 276 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 278 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 280 , the Indicator Feature Table or the Augmented Feature Table is used to train an artificial neural network classifier. The process can stop here at step 286 , or optionally, proceed to step 282 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 284 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0195": "Given the matrix of Boolean features, a na\u00efve bayes classifier may be trained using:", + "p0197": "And a new sample queried using:", + "p0199": "Where scores >0 are indicative of cancer. To avoid issues with numerical stability arising from a limited data set, a small positive value may be added to the numerator and denominator in each of the above ratios in order to ensure finite outputs.", + "p0200": "Linear Discriminant Analysis", + "p0201": "FIG. 8 illustrates the process of training a linear discriminate analysis classifier using an Indicator Feature Table or Augmented Indicator Feature Table.", + "p0202": "Reference is now being made to the flow diagram of FIG. 10 , which illustrates one embodiment of the present method for generating a classifier and/or multiclassifier comprising Indicator Features high resolution micro-ultrasound images. FIG. 10 illustrates the process of training a linear discriminate analysis classifier using an Indicator Feature Table or Augmented Indicator Feature Table. Flow processing begins at step 288 and immediately proceeds to step 290 . At step 290 , provide a set of Indicator Features. At step 292 , identify Indicator Features in each of the training images in an area of the image where the pathology is known, but blinded to the reader. At step 294 , the pathology data is linked to the sonographic features in the Indicator Feature Table because the tissue sample and training image are taken from the same physical location in the prostate. Optionally, additional information can be added to the Indicator Feature Table to generate an Augmented Feature Table. At step 296 , the Indicator Feature Table or the Augmented Feature Table is used to train a linear discriminate analysis classifier. The process can stop here at step 302 , or optionally, proceed to step 298 , to group the features based on the predictive probabilities of each Indicator Feature and/or combination of Indicator Features. At step 300 , form a new multiclass classifier based on the desired output scale of the classifier and the groupings of the features and the desired confidence intervals. Optionally, the classifier or the multiclass classifier can be further trained using either training data or test data.", + "p0203": "The threshold for classifying benign and cancerous tissue may be calculated using linear discriminant analysis, according to the following equations:", + "p0204": "w = \u22121", + "p0205": "c=w \u00bd", + "p0206": "Where all quantities are vectors in the N-dimensional feature space, \u03bc are the class means, \u03a3 are the class covariances, and the subscripts B and C are for the benign and cancerous classes respectively. The vector w then represents the normal to the hyperplane separating the two classes and scalar c the threshold along that hyperplane. A new observation may be classified by multiplying by w and then finding the distance and sign ) to the point c.", + "p0207": "Validating the Classifier", + "p0208": "Methods of validating a classifier are well known to one skilled in the art. In general, a classifier is validated by using it with a new independent set of images for which the pathology is known but not revealed to the person or computer reading the images to determine the presence of Indicator Features. The results are entered into an Indicator Feature Table and correlated with the pathology. This data is used to calculate ROC's for each reader. Accuracy is calculated by using the AUC under each reader's ROC.", + "p0209": "Exemplary Results", + "p0210": "Here are presented some preliminary results of a method according to an embodiment of the invention. FIG. 11 shows exemplary sonographic features used in an embodiment of a multiclass classifier.", + "p0211": "- FIG. 11A : Biopsy-proven benign tissue showing ductal proliferation. This regular ductal pattern was labeled as \u201cswiss cheese\u201d appearance. - FIG. 11B : Biopsy-proven benign tissue showing hyperechoic and ductal regions. The presence or absence of these scattered ducts was not found to significantly affect risk level. - FIG. 11C : This tissue was found to be benign on biopsy despite the mild hypoechoic region on the right. The tissue exhibits mild heterogeneity and includes small bright echos in otherwise hyperechoic tissue. - FIGS. 11D, 11E, and 11F These images show higher risk patterns, from left to right: - 11 D: Strong bright echos in hypoechoic tissue ; - 11 E: Heterogenous \u201csmudgy\u201d or \u201cmottled\u201d tissue , and - 11 F: Heterogenous \u201cCauliflower\u201d or - Focal Nodular Hyperplasia type tissue .", + "p0212": "All three biopsy samples were found to be malignant with Gleason Sums of 8, 7, and 8 respectively.", + "p0213": "- FIG. 11G : Irregular shadowing presents as vertical hypoechoic regions with no obvious source of attenuation . FIG. 11H : a mixed echo lesion with well-defined borders is present pressing on the rectal wall causing the prostate border to be irregular. In both cases the biopsy samples show a Gleason Sum of 9.", + "p0214": "A Database of High Resolution Prostate Images", + "p0215": "A database comprising high resolution micro-ultrasound prostate images, wherein Indicator Features in the images have been correlated to tissue biopsy results and optionally a risk score assigned by a classifier or multiclass classifier. The database is structured to: i) permit queries regarding co-occurrence of features, ii) examine various instances of each feature in order to train an automated analysis system such as a deep learning image analyzer to recognize the features; and iii) add new images, marked with indicator features so that the method can be continually improved by providing a better assessment of the predicted probability of cancer for each feature or combination of features.", + "p0216": "The database can optionally comprise additional data pertaining to the patient data obtained from, B-mode images, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies.", + "p0217": "System Implementations", + "p0218": "It is to be understood that embodiments of the present invention can be implemented in various forms of hardware, software, firmware, special purpose processes, or a combination thereof. In one embodiment, the present invention can be implemented in software as an application program tangible embodied on a computer readable program storage device. The application program can be uploaded to, and executed by, a machine comprising any suitable architecture.", + "p0219": "FIG. 14 is a block diagram of an exemplary computer system for implementing a method and system for classifying a patient's risk of having prostate cancer according to an embodiment of the invention. Referring now to FIG. 14 , a computer system 700 for implementing the present invention can comprise, inter alia, a central processing unit 702 , a memory 704 and an input/output interface 706 . The computer system 700 is generally coupled through the I/O interface 706 to a display 708 and various input devices 710 such as a mouse and a keyboard. The support circuits can include circuits such as cache, power supplies, clock circuits, and a communication bus. The memory 704 can include random access memory , read only memory , disk drive, tape drive, etc., or a combinations thereof. The present invention can be implemented as a routine 712 that is stored in memory 704 and executed by the CPU 702 to process the signal from the signal source 714 . As such, the computer system is a general purpose computer system that becomes a specific purpose computer system when executing the routine 712 of the present invention.", + "p0220": "The computer system 700 also includes an operating system and micro instruction code. The various processes and functions described herein can either be part of the micro instruction code or part of the application program which is executed via the operating system. In addition, various other peripheral devices can be connected to the computer platform such as an additional data storage device and a printing device.", + "p0221": "Flow Diagram of One Embodiment of Using the Classifier", + "p0222": "Reference is now being made to the flow diagram of FIG. 16 which illustrates one embodiment of the present method using the multiclass classifier for classifying patient risk for prostate cancer in accordance with the teachings hereof. Flow processing begins at step 400 and immediately proceeds to step 402 .", + "p0223": "At step 402 , receive a database of micro-ultrasound images containing Indicator Features and correlated to pathology results. At step 404 , use the system/method described above to train a classifier system or multi class classifier system, which functions to classify an unclassified patient into one of a plurality of risk levels. At step 406 , obtain the sonogram of a patient to be classified, and read the image for the presence of any of the features indicated by the classifier or multiclass classifier of 404 . This step may occur in real time during the biopsy procedure with a \u201clive\u201d image, or post hoc based on a saved image. In one embodiment, the features are: Small regular ducts, Hyperechoic tissue with our without ductal patches, mild heterogeneity, bright echoes in hyperechoic tissue, heterogeneous appearance, bright echoes, irregular peripheral zone, irregular prostate border, mixed-echo lesions, irregular shadowing.", + "p0224": "At step 408 , if a multiclass classifier is selected, use the multiclass classifier to classify the unclassified image into one of a plurality of classes. In an embodiment, this is accomplished by the operator consulting a printed flow chart or other visual aid generated based on step 404 which begins with high-risk features and proceeds to low risk features. In another embodiment, an automated system is used to classify the image.", + "p0225": "Reference is now being made to the flow diagram of FIG. 17 which is a continuation of the flow diagram of FIG. 16 with flow processing continuing with respect to Node B. At step 412 , communicate the image's classification to a display device. In other embodiments, the patient classification is communicated to any of: a storage device, a wireless handheld device, a laptop, tablet-PC, and a workstation. At step 414 , a determination is made whether the classified image suggests a significant risk for having prostate cancer.", + "p0226": "At step 416 , if the operator is conducting a live prostate biopsy, and the portion of tissue identified by the image in step 414 is determined to suggest a significant or elevated risk of prostate cancer, a targeted biopsy may be taken to confirm the diagnosis. Similarly, if images are being analyzed off-line in an active surveillance patient and the tissue identified by the image in step 414 is determined to have increased in risk level, the patient may be recalled so that a new biopsy sample may be taken, or potentially recommended for treatment.", + "p0227": "At step 420 , a determination is made whether to perform another classification. If so then processing repeats with respect to node D wherein, at step 408 , more images are obtained from this same patient. Processing repeats in a similar manner. If it is determined that further patient classification is not to be performed then, in this embodiment, further processing stops.", + "p0228": "At step 418 , once pathology results have been gathered for any biopsy or radical prostatectomy samples taken, the identified sonographic features and corresponding pathology may optionally be added to the database of step 402 . Step 404 may then be repeated to improve the classification system. It should be appreciated that the flow diagrams depicted herein are illustrative. One or more of the operations illustrated in the flow diagrams may be performed in a differing order. Other operations may be added, modified, enhanced, or consolidated. Variations thereof are intended to fall within the scope of the appended claims.", + "p0229": "Reference is now being made to FIG. 18 which shows a block diagram of one example system for performing various aspects of the present", + "p0230": "In FIG. 18 , a training set comprising images for a plurality of subjects are retrieved from a database 502 . Database 502 is a storage device wherein records are stored, manipulated, and retrieved in response to a query. Such records, in various embodiments, take the form of Tables of Indicator Features and/or Tables of Augmented Indicator Features. Although the database is shown as an external device, the database 502 may be internal to the workstation mounted, for example, on a hard disk therein. The training set is provided to the classifier system for training purposes. The features of the training set are obtained from the sonographic images", + "p0231": "In the embodiment shown, the classifier system comprises a plurality of modules. Learning Module 503 processes the training data contained in the records of the training set such that the classifier system can be trained. The Learning Module 503 further functions to prune the training set, as desired, such that the classifier is trained with data which meet a pre-determined criteria such as acceptable image quality. Once training has completed, Learning Module 503 signals Classification Module 504 to receive an image of a yet-to-be classified patient 506 . The unclassified patient's images 506 are received or are otherwise obtained by the classifier system which, in turn, proceeds to classify the unclassified image into one of a number of risk levels.", + "p0232": "Processor retrieves machine readable program instructions from Memory 505 and is provided to facilitate the functionality of the various modules comprising the classifier system. The processor, operating alone or in conjunction with other processors and memory, may be configured to assist or otherwise facilitate the functionality of any of the processors and modules of system.", + "p0233": "The classifier system of FIG. 18 is shown in communication with a workstation. A computer case of the workstation houses various components such as a motherboard with a processor and memory, a network card, a video card, a hard drive capable of reading/writing to machine readable media such as a floppy disk, optical disk, CD-ROM, DVD, magnetic tape, and the like, and other software and hardware needed to perform the functionality of a computer workstation. The workstation further includes a display device, such as a CRT, LCD, or touchscreen device, for displaying information, video, measurement data, computed values, medical information, results, locations, and the like. A user can view any of that information and make a selection from menu options displayed thereon. Keyboard and mouse effectuate a user input.", + "p0234": "It should be appreciated that the workstation has an operating system and other specialized software configured to display alphanumeric values, menus, scroll bars, dials, slideable bars, pull-down options, selectable buttons, and the like, for entering, selecting, modifying, and accepting information needed for processing in accordance with the teachings hereof. The workstation is further enabled to display the sonographic image and patient classifications as they are derived. The workstation may further display interim values, boundary conditions, and the like, in real-time as the classifier system performs its intended functionality as described herein in detail.", + "p0235": "A user or technician may use the user interface of the workstation to set parameters, view/adjust/delete values in the training set, and adjust various aspects of the classifier system as needed or as desired, depending on the implementation. Any of these selections or input may be stored/retrieved to storage device. Default settings can be retrieved from the storage device. A user of the workstation is also able to view or manipulate any of the records contained in the training set via pathways not shown.", + "p0236": "Although shown as a desktop computer, it should be appreciated that the workstation can be a laptop, mainframe, or a special purpose computer such as an ASIC, circuit, or the like. The embodiment of the workstation of FIG. 18 is illustrative and may include other functionality known in the arts. Any of the components of the workstation may be placed in communication with the classifier system or any devices in communication therewith. Any of the modules of the classifier system can be placed in communication with storage device and/or computer readable media and may store/retrieve therefrom data, variables, records, parameters, functions, and/or machine readable/executable program instructions, as needed to perform their intended functions. Each of the modules of the classifier system may be placed in communication with one or more remote devices over network.", + "p0237": "It should be appreciated that some or all of the functionality performed by any of the modules or processing units of the video processing system can be performed, in whole or in part, by the workstation placed in communication with the classifier system over network. The embodiment shown is illustrative and should not be viewed as limiting the scope of the appended claims strictly to that configuration. Various modules may designate one or more components which may, in turn, comprise software and/or hardware designed to perform the intended function.", + "p0238": "The teachings hereof can be implemented in hardware or software using any known or later developed systems, structures, devices, and/or software by those skilled in the applicable art without undue experimentation from the functional description provided herein with a general knowledge of the relevant arts. One or more aspects of the methods described herein are intended to be incorporated in an article of manufacture which may be shipped, sold, leased, or otherwise provided separately either alone or as part of a product suite or a service.", + "p0239": "The following clauses are offered as further description of the examples of the apparatus. Any one or more of the following clauses may be combinable with any another one or more of the following clauses and/or with any subsection or a portion or portions of any other clause and/or combination and permutation of clauses. Any one of the following clauses may stand on its own merit without having to be combined with any other clause or any portion of any other clause, etc. CLAUSE 1: A method of any clause in this paragraph, for generating one or more Indicator Features in high resolution ultrasound images, which can be used in assigning risk of prostate cancer during high resolution micro-ultrasound imaging the method comprising the steps of: i) generating a list of Possible Features in the images by: a) obtaining a high resolution micro-ultrasound image of prostate tissue that corresponds to tissue, which has also been biopsied and has been graded for a stage of cancer, ranging from benign to the highest grade; b) segmenting the region of the image corresponding to the biopsied tissue on the basis of contrasting areas or groups of pixels in the image, wherein each area or group of pixels constitutes a feature; and c) characterizing and providing a unique label to all detectable features and/or combination of features only in the region of the image that corresponds to biopsied tissue, ii) repeat steps a) through d) until all recurring patterns have been included in the list of Possible Features; iii) generate a Table of Candidate Features by: d) reading the micro-ultrasound images in the area corresponding to tissue, which has been graded for a stage of cancer, but for which the results are not known to the reader, identify and record in the Table the Candidate Features and/or combinations of Candidate Features observed in the area corresponding to graded tissue; e) correlating in the Table of Candidate Features the stage of cancer in the biopsied tissue that corresponds to the area in the image where each of the Candidate Features and/or combinations of Candidate Features are identified, wherein the stage ranges from benign to the highest grade; f) use the Table of Candidate Features as a training set of features to train a linear or nonlinear classifier to classify an Candidate Feature or combination of Candidate Features as cancerous or non-cancerous and thereby determine a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; g) eliminate Candidate Features or combinations of Candidate Features that do not yield predicted probabilities required for an Indicator Feature. CLAUSE 2: A method of any clause in this paragraph, wherein the areas are identified by analyzing groups of pixels. CLAUSE 3: A method of any clause in this paragraph, wherein the high resolution micro-ultrasound image is generated using an ultrasonic scanning device that can supply ultrasound at a center frequency of at least 15 MHz to 70 MHz. CLAUSE 4: A method of any clause in this paragraph, wherein the high resolution micro-ultrasound image is generated using an ultrasonic scanning device that can supply ultrasound at a center frequency of about 22 MHz. CLAUSE 5: A method of any clause in this paragraph, wherein the image is: B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhancement, and/or Spectral Analysis. CLAUSE 6: A method of any clause in this paragraph, wherein the biopsied tissue is obtained by: i) needle biopsy; ii) partial prostatectomy; iii) radical prostatectomy; or iv) a sample taken during focal therapy. CLAUSE 7: A method of any clause in this paragraph, wherein the reading is conducted by: i) a human; ii) a machine ; or iii) a human and a machine. CLAUSE 8: A method of any clause in this paragraph, wherein one or more Indicator Features obtained by the method of clause 7. CLAUSE 9: A method of any clause in this paragraph, wherein additional patient data is incorporated in the Table of Candidate Features and correlated with each Candidate Feature or combination of Candidate Features to the stage of cancer present in the tissue to generate an Augmented Table of Candidate Features, which is then used to train a non-linear classifier at step f). CLAUSE 10: A method of any clause in this paragraph, wherein the additional patient data comprises obtained from: spatial information about the image, spatial information about the features, 3-D information, spectral information about the image, and, images collected using other modalities such as B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarkers, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies. CLAUSE 11: A method of any clause in this paragraph, one or more Indicator Features obtained by the method of clause 10. CLAUSE 12: A method of any clause in this paragraph, including additional steps for generating a classifier, the method comprising the steps of: i) providing a Training Set of Indicator Features comprising Indicator Features and/or combinations of Indicator Features correlated to pathology of the tissue; ii) using the Training Set of Indicator Features to train a linear or nonlinear classifier that classifies a Indicator Features and/or combinations of Indicator Features as benign or cancerous and thereby determine a predicted probability of cancer for an Indicator Feature and/or combinations of Indicator Features. CLAUSE 13: A method of any clause in this paragraph, comprising the additional step of constructing a Risk Table comprising features and/or combinations of features correlated to their predicted probability. CLAUSE 14: A method of any clause in this paragraph, further comprising a Risk Table generated following the method of CLAUSE 13. CLAUSE 15: A method of any clause in this paragraph, wherein the linear classifier is: univariate relative risk, linear discriminate analyzer, or a na\u00efve Bayesian classifier. CLAUSE 16: A method of any clause in this paragraph, wherein the linear classifier calculates a univariate relative risk and a confidence interval are calculated for each Indicator Feature and/or combination of Indicator Features. CLAUSE 17: A method of any clause in this paragraph, wherein the non-linear classifier is a: binary classification tree, \u201crandom forest\u201d, support vector machine, or artificial neural network. CLAUSE 18: A method of any clause in this paragraph, wherein the classifier is executed by a computer comprising the steps of a patient image being read by the computer and processed to generate a predicted probability that the image relates to tissue that is cancerous. CLAUSE 19: A method of any clause in this paragraph, additionally comprising the step of including additional patient data into the Training Set of Indicator Features, wherein the additional risk parameters are correlated to an Indicator Feature or combination of Indicator Features. CLAUSE 20: A method of any clause in this paragraph, wherein the additional patient data comprises data obtained from: B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies. CLAUSE 21: A method of any clause in this paragraph, wherein the results of a high resolution micro-ultrasound image that has been classified using the classifier are combined into a risk table with additional data pertaining to the patient based on data obtained from: B-mode images, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies. CLAUSE 22: A method of any clause in this paragraph, wherein additional patient data is added to the risk table to generate an Augmented Table, which is then used to train a linear or non-linear classifier. CLAUSE 23: A method of any clause in this paragraph, additionally comprising steps to construct a multiclass classifier, wherein the additional steps are: grouping the Indicator Features and combinations of Indicator Features based on the predicted probabilities and confidence intervals determined by the classifier; and forming a new multiclass classifier by assigning a risk level to each grouping of Indicator Features, based on the desired output scale of the classifier. CLAUSE 24: A method of any clause in this paragraph, wherein the Indicator Features and/or combinations of Indicator Features are grouped based on the relative risk and confidence interval of each the Indicator Features and/or combinations of Indicator Features. CLAUSE 25: A method of any clause in this paragraph, wherein predefined thresholds are used to group features based on mean relative risk values. CLAUSE 26: A method of any clause in this paragraph, wherein a data set comprising balanced numbers of cancerous and benign observations, these predetermined values split the theoretical 0-2 range of the relative risk value into sections that provide useful assessment of the underlying risk. CLAUSE 27: A method of any clause in this paragraph, wherein the indicator features are assigned a risk level in the output scale based on their relative risk value according to the following categories: i) a mean RR range of 0.0-0.4 is assigned to be a very low risk; ii) a mean RR range of 0.4-0.6 is assigned to be some risk; iii) a mean RR range of 0.6-1.2 is assigned to be indeterminate risk; iv) a mean RR range of 1.2-1.6 is assigned to be significant risk; and v) a mean RR range of 1.6+ is assigned to be very high risk. CLAUSE 28: A method of any clause in this paragraph, wherein automated thresholding is used to determine the output scale. CLAUSE 29: A method of any clause in this paragraph, wherein the automated thresholding procedure comprises: i) choosing the pair of values with closest mean RR and comparing their CI ranges; ii) determining if the CI ranges overlap by a certain percentage, and if so, combining the values; iii) assigning the new mean for the group as the mean of the component RR values, and the CI as the intersection of the individual CI ranges; iv) repeat steps i) through iii) until no further merges are possible; v) repeat steps i) through iv) for the pair of values with the next closest mean RR values: vi) repeat steps i) through v) until every pair has been checked without a merge; viii) the resulting groups are sorted by their mean RR values and assigned Risk Scores from 1 to N . The overlap percentage may be adjusted to give a desirable number of classes for training purposes. CLAUSE 30: A method of any clause in this paragraph, wherein k-nearest neighbors clustering may be used with a fixed number of predefined starting classes spaced evenly over the RR space. CLAUSE 31: A method of any clause in this paragraph, wherein the multiclass classifier has between 3-10 possible risk levels. CLAUSE 32: A method of any clause in this paragraph, wherein the multiclass classifier comprises 5 possible risk levels of: a) very low risk for small regular ducts \u201cSwiss cheese\u201d with no other heterogeneity or bright echoes; b) some risk for hyperechoic, with or without ductal patches ; c) indeterminate risk for mild heterogeneity or bright echoes in hyperechoic tissue; d) significant risk for heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance or bright echoes; and e) very high risk for irregular shadowing originating within the prostate, not the prostate border, or mixed-echo lesions or irregular prostate and/or PZ border. CLAUSE 33: A method of any clause in this paragraph, a multiclass classifier generated according to the method of clause 32. CLAUSE 34: A method of any clause in this paragraph, wherein the multiclass classifier comprises 5 possible risk levels of: a) very low risk for small regular ducts or \u201cSwiss Cheese\u201d feature; b) some risk for hyperchoic with/without ductal patches features; c) indeterminate risk for a mild heterogeneity or bright echoes in hyperechoic tissue; d) significant risk for heterogeneous \u201ccauliflower, smudgy or mottled\u201d or bright echoes \u201cstarry sky\u201d features; and e) very high risk for irregular shadowing or mixed-echo lesions or irregular prostrate/PZ border. CLAUSE 35: A method of any clause in this paragraph, a multiclass classifier generated according to the method of clause 34. CLAUSE 36: A method of any clause in this paragraph, the method of using the multiclass classifier of clause 32 to assist in the diagnosis of prostate cancer. CLAUSE 37: A method of any clause in this paragraph, the method of using the multiclass classifier of clause 35 to assist in the diagnosis of prostate cancer. CLAUSE 38: A method of any clause in this paragraph, the method of using the multiclass classifier of clause 32 to direct a clinician as to where to take a biopsy core from the prostate. CLAUSE 39: A method of any clause in this paragraph, the method of using the multiclass classifier of clause 35 to direct a clinician as to where to take a biopsy core from the prostate. CLAUSE 40: A method of any clause in this paragraph, a plurality of Indicator Features generated according to the method of clause 7, comprising the same or similar characteristics as: a) small regular ducts \u201cSwiss cheese\u201d; b) hyperechoic with or without ductal patches; c) mild heterogeneity; d) bright echoes in hyperechoic tissue; e) heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance; f) bright echoes; g) irregular prostate ; h) irregular prostate border; i) mixed-echo lesions; or j) irregular shadowing. CLAUSE 41: A method of any clause in this paragraph, a plurality of Indicator Features generated according to the method of clause 7, comprising the same or similar characteristics as: a) regular ductal pattern \u201cSwiss cheese\u201d; b) hyperechoic with ductal patches; c) mild heterogeneity with small bright echoes in hyperechoic tissue; d) bright echoes; e) heterogeneous \u201csmudged/mottled\u201d; f) heterogeneous \u201ccauliflower\u201d; g) irregular shadowing; or h) mixed echo lesion causing irregular prostate borders. CLAUSE 42: A method of any clause in this paragraph, one or more Indicator Features having the same or similar characteristics as: i) small regular ducts \u201cSwiss cheese\u201d; ii) regular ductal pattern \u201cSwiss cheese\u201d; iii) hyperechoic with ductal patches; iv) hyperechoic with or without ductal patches; v) mild heterogeneity with small bright echoes in hyperechoic tissue; vi) mild heterogeneity; vii) bright echoes in hyperechoic tissue; viii) heterogeneous \u201csmudged/mottled\u201d; ix) heterogeneous \u201ccauliflower\u201d; x) heterogeneous \u201ccauliflower/smudgy/mottled\u201d appearance; xi) bright echoes; xii) irregular prostate ; xiii) irregular prostate border; xiv) mixed-echo lesions; xv) irregular shadowing; or xvi) mixed echo lesion causing irregular prostate borders. CLAUSE 43: A method of any clause in this paragraph, a method for generating a classifier, the method comprising the steps of: i) providing a Training Set of Indicator Features in high resolution micro-ultrasound images of prostate comprising Indicator Features and/or combinations of Indicator Features correlated to pathology of the tissue; ii) using the Training Set of Indicator Features to train a linear or nonlinear classifier that classifies a Indicator Features and/or combinations of Indicator Features as benign or cancerous and thereby determine a predicted probability of cancer for an Indicator Feature and/or combinations of Indicator Features. CLAUSE 44: A method of any clause in this paragraph, a classifier generated according to the method of clause 43. CLAUSE 45: A method of any clause in this paragraph, additionally comprising steps to construct a multiclass classifier, wherein the additional steps are: grouping the Indicator Features and combinations of Indicator Features based on the predicted probabilities and confidence intervals determined by the classifier; and forming a new multiclass classifier by assigning a risk level to each grouping of Indicator Features, based on the desired output scale of the classifier. CLAUSE 46: A method of any clause in this paragraph, a multiclass classifier generated according to the method of clause 45. CLAUSE 47: A program storage device of any clause in this paragraph, the program storage device readable by a computer tangibly embodying a program of instructions executable by the computer to perform the method steps for generating Indicator Features in high resolution micro-ultrasound prostate images, the method comprising the steps of: i) generating a list of Possible Features in the images by: a. obtaining a high resolution micro-ultrasound image of prostate tissue that corresponds to tissue, which has also been biopsied and has been graded for a stage of cancer, ranging from benign to the highest grade; b. segmenting the region of the image corresponding to the biopsied tissue on the basis of contrasting areas in the image, wherein each area constitutes a feature; and c. characterizing and providing a unique label to all detectable features and/or combination of features only in the region of the image that corresponds to biopsied tissue, ii) repeat steps a) through c) until all recurring patterns have been included in the list of Possible Features; iii) generate a Table of Candidate Features by: a) obtaining a training set of new high resolution micro-ultrasound image of prostate tissue that corresponds to tissue, which has also been subsequently biopsied and has been graded for a stage of cancer, ranging from benign to the highest grades; b) reading the micro-ultrasound images in the area corresponding to tissue, which has been graded for a stage of cancer, but for which the results are not known to the reader, identify and record in the Table the Candidate Features and/or combinations of Candidate Features observed in the area corresponding to graded tissue; c) correlating in the Table of Candidate Features the stage of cancer in the biopsied tissue that corresponds to the area in the image where each of the Candidate Features and/or combinations of Candidate Features are identified, wherein the stage ranges from benign to the highest grade; d) use the Table of Candidate Features as a training set of features to train a linear or nonlinear classifier to classify an Candidate Feature or combination of Candidate Features as cancerous or non-cancerous and thereby determine a predicted probability of cancer for a given Candidate Feature or combination of Candidate Features; e) eliminate Candidate Features or combinations of Candidate Features that do not yield predicted probabilities required for an Indicator Feature. CLAUSE 48: A program storage device of any clause in this paragraph, wherein the areas are identified by analyzing groups of pixels. CLAUSE 49: A program storage device of any clause in this paragraph, wherein the high resolution micro-ultrasound image is generated using an ultrasonic scanning device that can supply ultrasound at a center frequency of at least 15 MHz to 70 MHz. CLAUSE 50: A program storage device of any clause in this paragraph, wherein the high resolution micro-ultrasound image is generated using an ultrasonic scanning device that can supply ultrasound at a center frequency of about 22 MHz. CLAUSE 51: A program storage device of any clause in this paragraph, wherein the image is: B-mode , Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhancement, and/or Spectral Analysis. CLAUSE 52: A program storage device of any clause in this paragraph, the program storage device readable by a computer tangibly embodying a program of instructions executable by the computer to perform the method steps for generating a classifier, the method comprising the steps of: i) providing a Training Set of Indicator Features in high resolution micro-ultrasound images of prostate comprising Indicator Features and/or combinations of Indicator Features correlated to pathology of the tissue; ii) using the Training Set of Indicator Features to train a linear or nonlinear classifier that classifies a Indicator Features and/or combinations of Indicator Features as benign or cancerous and thereby determine a predicted probability of cancer for an Indicator Feature and/or combinations of Indicator Features. CLAUSE 53: A program storage device of any clause in this paragraph, configured to perform the additional method steps to construct a multiclass classifier, wherein the additional steps are: i) grouping the Indicator Features and combinations of Indicator Features based on the predicted probabilities and confidence intervals determined by the classifier; and ii) forming a new multiclass classifier by assigning a risk level to each grouping of Indicator Features, based on the desired output scale of the classifier. CLAUSE 54: A database of any clause in this paragraph, the database comprising high resolution micro-ultrasound prostate images, wherein Indicator Features in the images have been correlated to tissue biopsy results and optionally a risk score assigned by a classifier or multiclass classifier, wherein the database is structured to: i) permit queries regarding co-occurrence of features, and ii) examine various instances of each feature in order to train an automated analysis system such as a deep learning image analyzer to recognize the features iii) add new images, marked with indicator features so that the method can be continually improved by providing a better assessment of the predicted probability of cancer for each feature or combination of features. CLAUSE 55 A database of any clause in this paragraph, comprising additional data pertaining to the patient based on data obtained from: B-mode images, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhanced Ultrasound, Spectral Analysis, conventional resolution ultrasound, MRI, CT, or PET imaging, patient data comprising patient age, DRE results, PSA readings, free PSA, PSA density, family history, race, prostate volume, transition zone volume, relevant biomarker levels, protein/hormonal biomarker levels, genetic screening results, or number of prior negative biopsies.", + "p0240": "It will be appreciated that the above-disclosed and other features and functions, or alternatives thereof, may be desirably combined into other different systems or applications. Presently unforeseen or unanticipated alternatives, modifications, variations, or improvements may become apparent and/or subsequently made by those skilled in this art which are also intended to be encompassed by the following claims.", + "p0241": "It is to be further understood that, because some of the constituent system components and method steps depicted in the accompanying figures can be implemented in software, the actual connections between the systems components may differ depending upon the manner in which the present invention is programmed. Given the teachings of the present invention provided herein, one of ordinary skill in the related art will be able to contemplate these and similar implementations or configurations of the present invention.", + "p0242": "The teachings of any publications referenced herein are each hereby incorporated by reference in their entirety.", + "p0243": "While the present invention has been described in detail with reference to a preferred embodiment, those skilled in the art will appreciate that various modifications and substitutions can be made thereto without departing from the spirit and scope of the invention as set forth in the appended claims.", + "features": { + "F1": "A method for generating Indicator Features in high resolution ultrasound images to assign risk of prostate cancer, comprising obtaining a high resolution micro-ultrasound image of biopsied prostate tissue and surrounding tissue.", + "F2": "Segmenting a region of the image corresponding to the biopsied prostate tissue based on contrasting areas or groups of pixels, wherein each area or group of pixels constitutes a detectable feature.", + "F3": "Characterizing and labeling all detectable features and/or combinations of features in the biopsied region to create a list of Possible Features.", + "F4": "Repeating the steps of obtaining images, segmenting, and labeling until all detectable recurring patterns have been included in the list of Possible Features.", + "F5": "Generating a Table of Candidate Features by reading the micro-ultrasound images of graded prostate tissue to identify Candidate Features, which are a subset of Possible Features.", + "F6": "Recording the Candidate Features observed in areas corresponding to graded prostate tissue in the Table of Candidate Features.", + "F7": "Eliminating non-predictive Candidate Features by performing at least one of correlating stage of cancer to Candidate Features or using the Table as a training set for a classifier.", + "F8": "Eliminating Candidate Features with a predicted probability of cancer below a threshold value from the Table of Candidate Features.", + "F9": "High resolution micro-ultrasound image is generated using an ultrasonic scanning device at frequencies of 15 MHz to 70 MHz.", + "F10": "The image can be any one or a combination of B-mode, Velocity Color Flow Imaging, Power Doppler Color Flow Imaging, Elastography, Contrast Enhancement, or Spectral Analysis.", + "F11": "The reading of images is conducted by a human, a computer, or both.", + "F12": "The Indicator Features include a list of specific patterns like 'Swiss cheese' ducts, hyperechoic with ductal patches, and various heterogeneous features.", + "F13": "Additional patient data is incorporated into the Table of Candidate Features to generate an Augmented Table used for classifier training.", + "F14": "Training a classifier involves creating a Training Set of Indicator Features correlated to tissue pathology to classify features as benign or cancerous.", + "F15": "Including additional patient data correlated to Indicator Features in the Training Set for classifier training.", + "F16": "Steps to construct a multiclass classifier include grouping Indicator Features based on predicted probabilities and assigning risk levels.", + "F17": "A program storage device embodying instructions executable by a computer to perform the method steps for generating Indicator Features in high resolution micro-ultrasound prostate images." + } + } + }, + { + "FAN": "78286001", + "CPC": [ + "G01C-021/32", + "G01C-021/34", + "G01C-021/36/2", + "G01C-021/36/23", + "G01S-005/00/36", + "G06F-016/00", + "G06F-016/29", + "G06K-009/00/812", + "G06V-020/586", + "G07C-005/00/8", + "G08G-001/0968/16", + "G08G-001/0968/44", + "G08G-001/0968/66", + "G08G-001/0968/83", + "G08G-001/0969", + "G08G-001/14/3", + "G08G-001/14/4", + "G08G-001/14/6" + ], + "Content": { + "title": "Navigation method, device and system", + "pa01": "The embodiments of the present application provide a navigation method, device and system. The method is applied to a server. The method includes: receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and taking the position as an initial position; determining a position of one empty area of the areas as a target position based on position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database; and determining a first route from the initial position to the target position and displaying the first route on an electronic map of the mobile terminal. In embodiments of the present application, it is possible to provide the user with accurate navigation information and make him quickly find an empty parking space in a large parking lot.", + "c-en-0001": "A navigation method applied to a server, comprising: receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal.", + "c-en-0002": "The method according to claim 1 , wherein determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database comprises: displaying positions of respective empty areas on the electronic map of the mobile terminal based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, receiving one of the positions of respective empty areas sent by the mobile terminal, and determining this position as the target position; or determining positions of all current empty areas based on the position information corresponding to the respective areas and the correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database, and determining a position of one of the empty areas as the target position.", + "c-en-0003": "The method according to claim 1 , wherein after displaying the first route on an electronic map of the mobile terminal, the method further comprises: detecting a current position of the mobile terminal, and displaying the current position on the electronic map of the mobile terminal.", + "c-en-0004": "The method according to claim 3 , wherein detecting a current position of the mobile terminal comprises: obtaining a movement state of the mobile terminal based on an inertial sensor in the mobile terminal; and determining the current position of the mobile terminal based on the initial position of the mobile terminal and the movement state.", + "c-en-0005": "The method according to claim 4 , wherein after displaying the current position on the electronic map of the mobile terminal, the method further comprises: receiving current position feature information sent by the mobile terminal after receiving a position correction instruction sent by the mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and the database, and updating the current position of the mobile terminal with the position.", + "c-en-0006": "The method according to claim 3 , wherein detecting a current position of the mobile terminal comprises: receiving current position feature information periodically sent by the mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and the database, and taking the position as the current position.", + "c-en-0007": "The method according to claim 3 , wherein the method further comprises: detecting whether an offset between the current position and the first route is greater than a predetermined threshold; and if the offset is greater than the predetermined threshold, determining a second route from the current position to the target position based on the current position and the target position, and displaying the second route on the electronic map of the mobile terminal.", + "c-en-0008": "The method according to claim 1 , wherein the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number of a vehicle parked currently or an image of a vehicle parked currently; and the current position feature information includes at least one of: identification information of a parking space, a license plate number or a comparison image.", + "c-en-0009": "The method according to claim 1 , wherein for each area, the feature image of the area is acquired by an image acquiring device corresponding to the area after detecting a change of content included in the area.", + "c-en-0010": "A navigation method applied to a mobile terminal, comprising: obtaining current position feature information, and sending the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and receiving a route sent by the server and displaying the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database.", + "c-en-0011": "The method according to claim 10 , wherein after displaying the route on an electronic map, the method further comprises: receiving a current position of the mobile terminal sent by the server and displaying the current position on the electronic map of the mobile terminal.", + "c-en-0012": "The method according to claim 10 , wherein obtaining current position feature information comprises: receiving the current position feature information input by the user; or acquiring a feature image and taking the feature image as the current position feature information; or acquiring a feature image and extracting the current position feature information in the feature image, wherein the current position feature information includes at least one of an image including a preset area in the feature image and a character in the preset area.", + "c-en-0013": "The method according to claim 10 , wherein the database stores the identification information and the position information corresponding to each of parking spaces, and a license plate number of a vehicle parked currently or an image of a vehicle parked currently; and the current position feature information includes at least one of: identification information of a parking space, a license plate number or a comparison image.", + "c-en-0014": "A navigation system, comprising image acquiring devices installed in respective areas, a server and a mobile terminal, wherein the image acquiring devices are used to acquire feature images of corresponding areas; the server is used to perform a navigation method comprising: receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal; and the mobile terminal is used to perform a navigation method of comprising: obtaining the current position feature information, and sending the current position feature information to the server, so that the server determines the position of the area matching the current position feature information based on the current position feature information and the database stored therein, and takes the position as the initial position; and receiving a route sent by the server and displaying the route on the electronic map, wherein the route is a route from the initial position to the target position, which is determined by the server after determining the position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database.", + "c-en-0015": "An electronic device, comprising: a processor, a memory, a communication interface and a bus, wherein the processor, the memory and the communication interface are connected and communicated with each other via the bus; the memory stores executable program codes; and the processor executes a program corresponding to the executable program codes by reading the executable program codes stored in the memory to perform the navigation method of claim 1 .", + "c-en-0016": "An electronic device, comprising: a processor, a memory, a communication interface and a bus, wherein the processor, the memory and the communication interface are connected and communicated with each other via the bus; the memory stores executable program codes; and the processor executes a program corresponding to the executable program codes by reading the executable program codes stored in the memory to perform the navigation method of claim 10 .", + "p0001": "TECHNICAL FIELD", + "p0002": "The present application claims the priority to a Chinese patent application No. 201610522697.8 filed with the State Intellectual Property Office of People's Republic of China on Jun. 29, 2016 and entitled \u201cA navigation method, device and system\u201d, which is incorporated herein by reference in its entirety.", + "p0003": "TECHNICAL FIELD", + "p0004": "The present application relates to the field of navigation technology, and more particularly, to a navigation method, device and system.", + "p0005": "BACKGROUND", + "p0006": "With the rapid growth of the economy, vehicle population keeps on increasing, and the number of parking lots increases accordingly and the sizes thereof continuously increase. A large parking lot located in a place with large flows of people, on the one hand meets the need of driving and safely parking vehicles, but on the other hand makes the user easy to get lost therein because of the huge size of the site and the complex building structure. For example, when the user enters the parking lot, he/she cannot find an empty parking space.", + "p0007": "A relevant method for positioning a parking space can give the user an identification of an empty parking space when entering the parking lot to park, by installing parking cameras in the parking lot. Specifically, statuses of respective parking spaces in the entire parking lot can be displayed on the user's mobile terminal, so that the user can select one empty parking space.", + "p0008": "A parking camera is a monitoring camera installed in front of the parking space to photograph the status of the parking space. Specifically, it can be used to monitor whether every parking space is an empty parking space or not.", + "p0009": "The method for positioning a parking space described above can only inform the user of the identification of the empty parking space that the user wants to find, and cannot tell the user how to reach the empty parking space or inform the user of his/her current position. In a large parking lot, even if the user knows the identification of the empty parking space, it is difficult for the user to find the empty parking space. Therefore, how to provide the user with accurate navigation information to make him find the empty parking space in a large parking lot quickly is a problem to be solved.", + "p0010": "SUMMARY", + "p0011": "The purpose of the embodiments of the present application is to provide a navigation method, device and system, so as to provide the user with accurate navigation information to make him quickly find an empty parking space in a large parking lot. Specific technical solutions are as follows.", + "p0012": "In a first aspect, an embodiment of the present application provides a navigation method applied to a server, the method comprising:", + "p0013": "receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0014": "determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0015": "determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal.", + "p0016": "Optionally, determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database includes:", + "p0017": "displaying positions of respective empty areas on the electronic map of the mobile terminal based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, receiving one of the positions of respective empty areas sent by the mobile terminal, and determining this position as the target position; or", + "p0018": "determining positions of all current empty areas and determining a position of one of the empty areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0019": "Optionally, after displaying the first route on an electronic map of the mobile terminal, the method further includes:", + "p0020": "detecting a current position of the mobile terminal, and displaying the current position on the electronic map of the mobile terminal.", + "p0021": "Optionally, detecting a current position of the mobile terminal includes:", + "p0022": "obtaining a movement state of the mobile terminal based on an inertial sensor in the mobile terminal; and", + "p0023": "determining the current position of the mobile terminal based on the initial position of the mobile terminal and the movement state.", + "p0024": "Optionally, after displaying the current position on an electronic map of the mobile terminal, the method further includes:", + "p0025": "receiving current position feature information sent by the mobile terminal after receiving a position correction instruction sent by the mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and the database, and updating the current position of the mobile terminal with this position.", + "p0026": "Optionally, detecting a current position of the mobile terminal includes:", + "p0027": "receiving current position feature information periodically sent by the mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and the database, and taking the position as the current position.", + "p0028": "Optionally, the method further includes:", + "p0029": "detecting whether an offset between the current position and the first route is greater than a predetermined threshold; and", + "p0030": "if the offset is greater than the predetermined threshold, determining a second route from the current position to the target position based on the current position and the target position, and displaying the second route on the electronic map of the mobile terminal.", + "p0031": "Optionally, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently, and the current position feature information includes at least one of the identification information of the parking space, a license plate number or a comparison image.", + "p0032": "Optionally, for each area, a feature image of the area is acquired by an image acquiring device corresponding to the area after detect a change of content included in the area.", + "p0033": "In a second aspect, an embodiment of the present application provides a navigation method applied to a mobile terminal, the method comprising:", + "p0034": "obtaining current position feature information, and sending the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and", + "p0035": "receiving a route sent by the server and displaying the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0036": "Optionally, after displaying the route on an electronic map, the method further includes:", + "p0037": "receiving a current position of the mobile terminal sent by the server and displaying the current position on the electronic map of the mobile terminal.", + "p0038": "Optionally, obtaining current position feature information includes:", + "p0039": "receiving the current position feature information input by the user;", + "p0040": "acquiring a feature image and taking the feature image as the current position feature information; or", + "p0041": "acquiring a feature image and extracting the current position feature information in the feature image, wherein the current position feature information includes at least one of an image including a preset area in the feature image and a character in the preset area.", + "p0042": "Optionally, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently, and the current position feature information includes at least one of the identification information of the parking space, a license plate number or a comparison image.", + "p0043": "In a third aspect, an embodiment of the present application provides a navigation device applied to a server, the device comprising:", + "p0044": "a first processing module used to receive current position feature information sent by a mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and take the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0045": "a determining module used to determine a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0046": "a second processing module used to determine a first route from the initial position to the target position, and display the first route on an electronic map of the mobile terminal.", + "p0047": "Optionally, the determining module is specifically used to display positions of respective empty areas on the electronic map of the mobile terminal based on position information corresponding to the respective areas and correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database, receive one of the positions of respective empty areas sent by the mobile terminal, and determine this position as the target position; or", + "p0048": "determine positions of all current empty areas based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, and determine a position of one of the empty areas as the target position.", + "p0049": "Optionally, the device further includes:", + "p0050": "a detecting module used to detect a current position of the mobile terminal, and display the current position on the electronic map of the mobile terminal.", + "p0051": "Optionally, the detecting module includes:", + "p0052": "an obtaining sub-module used to obtain a movement state of the mobile terminal based on an inertial sensor in the mobile terminal; and", + "p0053": "a determining sub-module used to determining a current position of the mobile terminal based on the initial position of the mobile terminal and the movement state.", + "p0054": "Optionally, the device further includes:", + "p0055": "a third processing module used to receive the current position feature information sent by the mobile terminal when receiving a position correction instruction sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database, and update the current position of the mobile terminal with this position.", + "p0056": "Optionally, the detecting module is specifically used to receive the current position feature information periodically sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database, and take the position as the current position.", + "p0057": "Optionally, the detecting module is also used to detect whether an offset between the current position and the first route is greater than a predetermined threshold; and the device further includes:", + "p0058": "a fourth processing module used to determine a second route from the current position to the target position based on the current position and the target position, and display the second route on the electronic map of the mobile terminal, when the detecting module detects that the offset between the current position and the first route is greater than the predetermined threshold.", + "p0059": "Optionally, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently, and the current position feature information includes at least one of identification information of a parking space, a license plate number or a comparison image.", + "p0060": "Optionally, for each area, a feature image of the area is acquired by an image acquiring device corresponding to the area after detecting a change of content included in the area.", + "p0061": "In a fourth aspect, an embodiment of the present application provides a navigation device applied to a mobile terminal, the device comprising:", + "p0062": "a first processing module used to obtain current position feature information and send the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and", + "p0063": "a second processing module used to receive a route sent by the server and display the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0064": "Optionally, the device further includes:", + "p0065": "a receiving module used to receive a current position of the mobile terminal sent by the server and display the current position on the electronic map of the mobile terminal.", + "p0066": "Optionally, the first processing module is specifically used to receive current position feature information input by the user;", + "p0067": "acquire a feature image and take the feature image as the current position feature information; or", + "p0068": "acquire a feature image and extract current position feature information in the feature image, wherein the current position feature information includes at least one of an image including a preset area in the feature image and a character in the preset area.", + "p0069": "Optionally, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently, and the current position feature information includes at least one of identification information of a parking space, a license plate number, or a comparison image.", + "p0070": "In a fifth aspect, the embodiment of the present application provides a navigation system including image acquiring devices installed in respective areas, a server and a mobile terminal, wherein,", + "p0071": "the image acquiring devices are used to acquire feature images of corresponding areas;", + "p0072": "the server is used to perform the navigation method as described in the first aspect;", + "p0073": "the mobile terminal is used to perform the navigation method as described in the second aspect.", + "p0074": "In a sixth aspect, the embodiment of the present application provides a navigation system including image acquiring devices installed in respective areas, a navigation device located in a server as described in the third aspect, and a navigation device located in a mobile terminal as described in the fourth aspect.", + "p0075": "In a seventh aspect, the present application provides a storage medium for storing executable program codes, which are used to perform the navigation method described in the first aspect when being executed.", + "p0076": "In an eighth aspect, the present application provides a storage medium for storing executable program codes, which are used to perform the navigation method described in the second aspect when being executed.", + "p0077": "In a ninth aspect, the present application provides an application for performing the navigation method described in the first aspect when being executed.", + "p0078": "In a tenth aspect, the present application provides an application for performing the navigation method described in the second aspect when being executed.", + "p0079": "In an eleventh aspect, the present application provides an electronic device comprising:", + "p0080": "a processor, a memory, a communication interface and a bus,", + "p0081": "wherein, the processor, the memory and the communication interface are connected and communicated with each other via the bus;", + "p0082": "the memory stores executable program codes; and", + "p0083": "the processor executes a program corresponding to the executable program codes by reading the executable program codes stored in the memory to perform the navigation method as described in the first aspect.", + "p0084": "In a twelfth aspect, the present application provides an electronic device comprising:", + "p0085": "a processor, a memory, a communication interface and a bus,", + "p0086": "wherein the processor, the memory and the communication interface are connected and communicated with each other via the bus;", + "p0087": "the memory stores executable program codes; and", + "p0088": "the processor executes a program corresponding to the executable program codes by reading the executable program codes stored in the memory to perform the navigation method as described in the second aspect.", + "p0089": "The embodiment of the present application provides a navigation method, device and system. A server can accurately determine an initial position based on current position feature information sent by a mobile terminal and a database stored in the server, and can determine a position of one empty area in the database as a target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on an electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0090": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0091": "In order to explain the technical solutions of embodiments of the present invention or of the prior art more clearly, the accompanying figures required to be used in the description of the embodiments or the prior art will be described briefly below. Obviously, the figures described below are just some embodiments of the present invention and other figures can be obtained by those skilled in the art based on these figures without any creative efforts.", + "p0092": "FIG. 1 is a flow chart of a navigation method provided according to an embodiment of the present application;", + "p0093": "FIG. 2 is another flow chart of a navigation method provided according to an embodiment of the present application.", + "p0094": "FIG. 3 is another flow chart of a navigation method provided according to an embodiment of the present application.", + "p0095": "FIG. 4 is another flow chart of a navigation method provided according to an embodiment of the present application.", + "p0096": "FIG. 5 is another flow chart of a navigation method provided according to an embodiment of the present application.", + "p0097": "FIG. 6 is a schematic structural view of a navigation device provided according to an embodiment of the present application.", + "p0098": "FIG. 7 is another schematic structural view of a navigation device provided according to an embodiment of the present application.", + "p0099": "FIG. 8 is another schematic structural view of a navigation device provided according to an embodiment of the present application.", + "p0100": "FIG. 9 is another schematic structural view of a navigation device provided according to an embodiment of the present application.", + "p0101": "FIG. 10 is a schematic structural view of a navigation system provided according to an embodiment of the present application.", + "p0102": "DETAILED DESCRIPTION", + "p0103": "In order to make those skilled in the art better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying figures of embodiments of the present invention. Obviously, the embodiments described are merely some of, not all of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without any creative efforts shall fall within the scope of protection of the present application.", + "p0104": "It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings in conjunction with the embodiments.", + "p0105": "In the embodiment of the present application, in order to provide the user with accurate navigation information and improve the user experience, a database can be created in advance in a server. The database can be created by the server according to correspondence relationship between respective feature information and identification information of the respective areas as well as position information corresponding to the respective areas. An image acquiring device corresponding to each area acquires a feature image of this corresponding area, and obtains corresponding feature information based on the feature image and then sends the corresponding feature information to the server.", + "p0106": "For example, when the navigation information in a parking lot is provided for the user, identification information, position information, and feature information of respective parking spaces in the parking lot can be stored in the database. When a parking space is in an empty state currently, the feature information thereof indicates the empty state. When the parking space is in a non-empty state currently, the feature information thereof is information of a vehicle parked in this parking space currently, such as a license plate number, image of the vehicle and the like.", + "p0107": "The database stored in the server can be shown as the following table:", + "p0108": "Identification Position Feature information information information 1 x1, y1 Idle 2 x2, y2 JING Nxxx 3 x3, y3 JIN Axxx . . . . . . . . . N xn, yn Idle", + "p0109": "When an area is not empty, corresponding feature information thereof can be stored in the form of text, and can also be stored in the form of an image.", + "p0110": "In order to provide the user with accurate navigation information and improve the user experience, the present application provides a navigation method applied to a server. As shown in FIG. 1 , the method can include steps S 101 -S 103 .", + "p0111": "In S 101 , current position feature information sent by a mobile terminal is received, and a position of an area matching the current position feature information is determined based on the current position feature information and a database stored in the server, and the position is taken as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas.", + "p0112": "In the embodiment of the present application, when entering a parking lot, a user uses his/her mobile terminal to obtain the navigation information through a server corresponding to the parking lot.", + "p0113": "Specifically, the user can input the current position feature information through the mobile terminal. Further, the mobile terminal can send the current position feature information to the server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and the database stored therein, and takes the position as an initial position.", + "p0114": "The current position feature information input by the user through the mobile terminal can be at least one of a parking space identification, a license plate number, a comparison image or the like.", + "p0115": "For example, when entering a parking lot, the user can input identification of a parking space which he/she is on currently into the mobile terminal as the current position feature information; or, when there is a vehicle parked at the user's current position, the user can input the license plate number of the vehicle as the current position feature information; or, the user can also take a feature image at the his/her current position by a photographing function of the mobile terminal and obtain the current position feature information based on the feature image.", + "p0116": "When obtaining the feature image, the mobile terminal can send directly the feature image to the server as the current position feature information. Alternatively, the mobile terminal can perform a feature extraction on an acquired feature image to extract the feature information therein, and send it to the server as the current position feature information. When the mobile terminal extracts the feature information in the feature image, the feature information extracted can be an image, and can be also a text.", + "p0117": "For example, the mobile terminal can use any one of the existing image analysis methods to identify the feature information such as a parking space identification and a license plate number in the feature image, and send the feature information extracted to the server in the form of text as the current position feature information. Alternatively, when identifying the feature information such as the parking space identification and the license plate number in the feature image, the mobile terminal can obtain an image of an area containing the feature information and send the image to the server as the current position feature information.", + "p0118": "Any of the existing methods can be employed in the process of extracting the feature information from the feature image by the mobile terminal, which will not be described in the embodiment of the present application.", + "p0119": "After obtaining the current position feature information, the mobile terminal can further send the current position feature information to the server. After receiving the current position feature information sent by the mobile terminal, the server can search a position of an area matching the current position feature information in the database stored therein. Since the initial position feature information input by the user is the information obtained at his/her current position, the position determined by the server is the current position of the user, that is, the initial position for determining a navigation route.", + "p0120": "For example, when the user inputs a parking space identification, the server can search a position of a parking space of this identification in the database and take the position as an initial position. When the user inputs a license plate number, the server can search a position of a vehicle of this license plate number in the database and take the position as an initial position.", + "p0121": "In S 102 , a position of one empty area is determined as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0122": "After determining the initial position, the server can further determine the target position. Specifically, according to different needs, the target position can be an empty parking space or a position where the user's vehicle is parked.", + "p0123": "When the user wants to park, in an implementation, the server can determine positions of all current empty areas based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, and determine a position of one of the empty areas as the target position.", + "p0124": "In another implementation, in order to improve the user experience, the server can display all the empty areas on an electronic map of the mobile terminal based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, and one of the empty areas can be selected as a target position by the user. For example, the user can select a position closer to him/her through the mobile terminal. The mobile terminal sends this position to the server, and the server can directly take this position as the target position.", + "p0125": "It should be noted that when the server displays all the empty areas on the electronic map, only the empty areas can be displayed. Alternatively, in order to make the user know a parking condition of the entire parking lot, non-empty areas can also be displayed on the electronic map, and the empty areas and the non-empty areas are marked out. Further, the user can select one of the empty areas, and take the position thereof as the target position.", + "p0126": "When the user wants to find his/her parked vehicle, the target location is the position of the user's parked vehicle. In this case, the user can input the target position feature information through the mobile terminal. The target position feature information can be at least one of a parking space identification, a license plate number or a comparison image.", + "p0127": "For example, when the user wants to find his/her parked vehicle, he/she can input the license plate number or vehicle image of his/her vehicle, or when the user remembers the parking space identification when parking, he/she can input the parking space identification as target position feature information.", + "p0128": "After the user inputs the target position feature information to the mobile terminal, the mobile terminal can further send the target position feature information to the server. After receiving the target position feature information sent by the mobile terminal, the server can search a position of an area matching the target position feature information in the database stored therein, and take the position as the target position.", + "p0129": "In S 103 , a first route from the initial position to the target position is determined, and the first route is displayed on an electronic map of the mobile terminal.", + "p0130": "After determining the target position and the initial position, the server can determine the first route from the initial position to the target location. For example, the server can take the nearest route among various routes from the initial position to the target position as the first route based on a route distribution in the parking lot.", + "p0131": "Any of the existing techniques can be employed in the process of determining the first route from the initial position to the target position by the server, which will not be described in the embodiment of the present application.", + "p0132": "After determining the first route from the initial position to the target position, the server can send the first route to the mobile terminal and display the first route on the electronic map of the mobile terminal.", + "p0133": "For example, the server can display an electronic map of the parking lot on the mobile terminal and display the first route determined on the electronic map.", + "p0134": "The embodiment of the present application provides a navigation method. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine the route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0135": "As an implementation of the embodiment of the present application, in order to make the server create an accurate database, a corresponding image acquiring device can be configured in each area. The image acquiring device can acquire a feature image of a corresponding area, and can obtain corresponding feature information based on the feature image and send it to the server. Further, the server can create a database based on correspondence relationship between respective feature information and respective areas, as well as positions of respective areas.", + "p0136": "For example, a corresponding image acquiring device can be configured at each area, or when the range of the area is relatively small and an image acquiring area of one image acquiring device can include a plurality of areas, one image acquiring device can correspond to a plurality of areas. Respective image acquiring devices can periodically acquire feature images of the corresponding areas in a set time interval, and can obtain corresponding feature information based on the feature images and send it to the server.", + "p0137": "In one implementation, the image acquiring device can send the acquired feature image directly to the server as the feature information. In another implementation, the image acquiring device can perform feature extraction on the acquired feature image to extract feature information therein, and send the feature information to the server.", + "p0138": "For example, the image acquiring device can use any of the existing image analysis methods to identify the feature information such as a parking space identification and a license plate number in the feature image, and send the feature information extracted to the server in the form of text as the current position feature information. Alternatively, when identifying the feature information such as the parking space identification and the license plate number in the feature image, the mobile terminal can obtain an image of an area containing the feature information and send the image to the server as the current position feature information.", + "p0139": "Any of the existing method can be employed in the process of extracting the feature information from the feature image by the image acquiring device, which will not be described in the embodiment of the present application.", + "p0140": "Optionally, respective image acquiring devices can detect whether or not the content contained in corresponding areas thereof is changed so as to avoid the excessive load of the image acquiring devices caused by periodically acquiring the feature images. For example, the image acquiring device can detect whether the corresponding area thereof is changed from an empty state to a state that a vehicle is parked therein, or detect whether the vehicle parked in the corresponding area is changed.", + "p0141": "When the image acquiring device detects that the content contained in the corresponding area is changed, it can acquire a feature image of the area. That is to say, the image acquiring device can acquire a feature image corresponding to a changed area, thereby ensuring the accuracy of the database stored in the server while the load pressure of respective image acquiring devices is reduced.", + "p0142": "As an implementation of the embodiment of the present application, as shown in FIG. 2 , the navigation method provided by the present application, after step S 103 , can further include step S 104 .", + "p0143": "In S 104 , a current position of the mobile terminal is detected, and the current position is displayed on the electronic map of the mobile terminal.", + "p0144": "In the embodiment of the present application, after determining the first route from the initial position to the target position and displays the first route on the electronic map of the mobile terminal, the server can also detect a current position of the mobile terminal and display the current position on the electronic map of the mobile terminal.", + "p0145": "In one implementation, the server can obtain a movement state of the mobile terminal, such as acceleration, angular acceleration and the like of the mobile terminal, based on an inertial sensor in the mobile terminal, such as an accelerometer, a gyroscope and the like, and then can determine the current position of the mobile terminal based on the initial position obtained previously and the movement state of the mobile terminal.", + "p0146": "When the server determines the current position of the mobile terminal based on the inertial sensor in the mobile terminal, in some cases, the determined position can be inaccurate.", + "p0147": "Thus, in the present embodiment, the user can determine whether the current position displayed on the mobile terminal is correct based on the current position displayed on the mobile terminal and a current actual position of the user. When determining that the current position displayed on the mobile terminal has a relatively large difference from the position of the user, the user can perform position correction.", + "p0148": "Specifically, the user can input a position correction instruction in the mobile terminal. The mobile terminal can send the position correction instruction to the server when receiving the position correction instruction input by the user. Additionally, the user can input current position feature information again through the mobile terminal, and further, the mobile terminal can send the current position feature information to the server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and the database stored therein, and updates the current position of the mobile terminal with this position.", + "p0149": "The process of inputting current position feature information by the user through the mobile terminal and the process of determining a position of an area matching the current position feature information by the server based on the current position feature information and the database are similar to the above process of determining the initial position and will not be described in the embodiment of the present application.", + "p0150": "In another implementation, in order to improve the accuracy of determining the current position, the server can receive current position feature information periodically sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database stored in the server, take the position as the current position, and display the current position on the electronic map of the mobile terminal.", + "p0151": "Any of the existing methods can be also employed in the process of determining the current position by the server, which will not be described in the embodiment of the present application.", + "p0152": "By detecting the current position of the mobile terminal and displaying the current position on the electronic map of the mobile terminal, the user can obtain his/her real-time position to determine whether his/her route is correct.", + "p0153": "As an implementation of the embodiment of the present application, as shown in FIG. 3 , the navigation method provided by the present application, after step S 104 , can further include steps S 105 -S 106 .", + "p0154": "In S 105 , it is detected whether an offset between the current position and the first route is greater than a predetermined threshold.", + "p0155": "In some cases, when the server displays the first route on the mobile terminal, the route of the user can also deviate from the first route.", + "p0156": "In the embodiment of the present application, the server can detect whether the offset between the current position of the mobile terminal and the first route is greater than a predetermined threshold, such as 3 m, 5 m, 10 m, and the like.", + "p0157": "Specifically, the user can compare whether a minimum distance between the current position of the mobile terminal and the first route is greater than the predetermined threshold based on the first route and the current position of the mobile terminal. If the minimum distance is greater than the predetermined threshold, the user determines whether the offset between the current position and the first route is greater than the predetermined threshold.", + "p0158": "In S 106 , when the detected offset between the current position and the first route is greater than the predetermined threshold, a second route from the current position to the target position is determined based on the current position and the target position, and the second route is displayed on the electronic map of the mobile terminal.", + "p0159": "When determining that the offset between the current position of the mobile terminal and the first route is greater than the predetermined threshold, the server can determine a second route from the current position to the target position based on the current position and the target position, and display the second route on the electronic map of the mobile terminal.", + "p0160": "Any of the existing techniques can be employed in the process of determining the second route from the current position to the target position by the server, which will not be described in the embodiment of the present application.", + "p0161": "In the present scheme, when determining that the offset between the current position of the mobile terminal and the first route is greater than the predetermined threshold, the server can re-plan a route based on the current position of the mobile terminal and the target position, thereby ensuring that the user smoothly reach the target position and improving the user experience.", + "p0162": "In order to provide the user with accurate navigation information and improve the user experience, the present application provides a navigation method applied to a mobile terminal. As shown in FIG. 4 , the method can include steps S 201 -S 202 .", + "p0163": "In S 201 , current position feature information is obtained and sent to a server, so then the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas.", + "p0164": "In the embodiment of the present application, when entering a parking lot, the user can use his/her mobile terminal to obtain the current position feature information. Further, the mobile terminal can send the current position feature information to the server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and the database stored therein, and takes the position as an initial position.", + "p0165": "The current position feature information input by the user through the mobile terminal can be at least one of a parking space identification, a license plate number, a comparison image or the like.", + "p0166": "For example, when entering a parking lot, the user can input the identification of a parking space which he/she is on currently into the mobile terminal as the current position feature information; or, when there is a vehicle parked at the user's current position, he/she can input the license plate number of the vehicle as the current position feature information; or, the user can also take a feature image at the his/her current position by a photographing function of the mobile terminal and obtain the current position feature information based on the feature image.", + "p0167": "When obtaining the feature image, the mobile terminal can send directly the feature image to the server as the current position feature information. Alternatively, the mobile terminal can perform a feature extraction on an acquired feature image to extract the feature information therein, and send it to the server as the current position feature information. When the mobile terminal extracts the feature information in the feature image, the feature information extracted can be an image, and can be also a text.", + "p0168": "For example, the mobile terminal can use any one of the existing image analysis methods to identify the feature information such as a parking space identification and a license plate number in the feature image, and send the feature information extracted to the server in the form of text as the current position feature information. Alternatively, when identifying the feature information such as the parking space identification and the license plate number in the feature image, the mobile terminal can obtain an image of an area containing the feature information and send the image to the server as the current position feature information.", + "p0169": "Any of the existing methods can be employed in the process of extracting the feature information from the feature image by the mobile terminal, which will not be described in the embodiment of the present application.", + "p0170": "After obtaining the current position feature information, the mobile terminal can further send the current position feature information to the server. After receiving the current position feature information sent by the mobile terminal, the server can search a position of an area matching the current position feature information in the database stored therein. Since the initial position feature information input by the user is the information obtained at his/her current position, the position determined by the server is the current position of the user, that is, the initial position for determining a navigation route.", + "p0171": "For example, when the user inputs a parking space identification, the server can search a position of a parking space of this identification in the database and take the position as an initial position. When the user inputs a license plate number, the server can search a position of a vehicle of this license plate number in the database and take the position as the initial position.", + "p0172": "In S 202 , a route sent by the server is received and displayed on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0173": "In the embodiment of the present application, after determining the initial position, the server can further determine the target position. Specifically, according to different needs, the target position can be an empty parking space or a position where the user's vehicle is parked.", + "p0174": "When the user wants to park, in an implementation, the server can determine positions of all current empty areas based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, and determine a position of one of the empty areas as the target position.", + "p0175": "In another implementation, in order to improve the user experience, the server can display all the empty areas on an electronic map of the mobile terminal based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database, and one of the empty areas can be selected by the user as a target position. For example, the user can select a position closer to him/her through the mobile terminal. The mobile terminal sends this position to the server, and the server can directly take this position as the target position.", + "p0176": "It should be noted that when the server displays all the empty areas on the electronic map, only the empty areas can be displayed. Alternatively, in order to make the user know a parking condition of the entire parking lot, non-empty areas can also be displayed on the electronic map, and the empty areas and the non-empty areas are marked out. Further, the user can select one of the empty areas, and take the position thereof as the target position.", + "p0177": "When the user wants to find his/her parked vehicle, the target location is the position of the user's parked vehicle. In this case, the user can input the target position feature information through the mobile terminal. The target position feature information can be at least one of a parking space identification, a license plate number or a comparison image.", + "p0178": "For example, when the user wants to find his/her parked vehicle, he/she can input the license plate number or vehicle image of his/her vehicle, or when the user remembers the parking space identification while parking, he/she can input the parking space identification as target position feature information.", + "p0179": "After the user inputs the target position feature information into the mobile terminal, the mobile terminal can further send the target position feature information to the server. After receiving the target position feature information sent by the mobile terminal, the server can search a position of an area matching the target position feature information in the database stored therein, and take the position as the target position.", + "p0180": "After determining the target position and the initial position, the server can determine a route from the initial position to the target location. Any of the existing techniques can be employed in the process of determining the route from the initial position to the target position by the server, which will not be described in the embodiment of the present application.", + "p0181": "After determining the route from the initial position to the target position, the server can send the route to the mobile terminal and display the route on the electronic map of the mobile terminal.", + "p0182": "The embodiment of the present application provides a navigation method. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine the route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0183": "As an implementation of the embodiment of the present application, as shown in FIG. 5 , the navigation method provided by the present application, after step S 202 , can include step S 203 .", + "p0184": "In S 203 , a current position of the mobile terminal sent by the server is received and the current position is displayed on the electronic map of the mobile terminal.", + "p0185": "In the embodiment of the present application, after determining the route from the initial position to the target position and displaying the route on the electronic map of the mobile terminal, the server can also detect a current position of the mobile terminal and send the current position to the mobile terminal.", + "p0186": "In an implementation, the server can obtain a movement state of the mobile terminal, such as acceleration, angular acceleration and the like of the mobile terminal, based on an inertial sensor in the mobile terminal, such as an accelerometer, a gyroscope and the like, and then can determine the current position of the mobile terminal based on the initial position obtained previously and the movement state of the mobile terminal.", + "p0187": "When the server determines the current position of the mobile terminal based on the inertial sensor in the mobile terminal, in some cases, the determined position can be inaccurate.", + "p0188": "Thus, in the present embodiment, the user can determine whether the current position displayed on the mobile terminal is correct based on the current position displayed on the mobile terminal and a current actual position of the user. When determining that the current position displayed on the mobile terminal has a relatively large difference from the position of the user, the user can perform position correction.", + "p0189": "Specifically, the user can input a position correction instruction in the mobile terminal. The mobile terminal can send the position correction instruction to the server when receiving the position correction instruction input by the user. Additionally, the user can input current position feature information again through the mobile terminal, and further, the mobile terminal can send the current position feature information to the server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and the database stored therein, and updates the current position of the mobile terminal with this position.", + "p0190": "The process of inputting current position feature information by the user through the mobile terminal and the process of determining a position of an area matching the current position feature information by the server based on the current position feature information and the database are similar to the above process of determining the initial position and will not be described in the embodiment of the present application.", + "p0191": "In another implementation, in order to improve the accuracy of determining the current position, the server can receive current position feature information periodically sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database stored in the server, take the position as the current position, and display the current position on the electronic map of the mobile terminal.", + "p0192": "Any of the existing methods can be also employed in the process of determining the current position by the server, which will not be described in the embodiment of the present application.", + "p0193": "Therefore, the mobile terminal can receive the current position sent by the server and display the current position on the electronic map thereof.", + "p0194": "By detecting the current position of the mobile terminal and displaying the current position on the electronic map of the mobile terminal, the user can obtain his/her real-time position to determine whether his/her route is correct.", + "p0195": "An embodiment of the present application also provides a device embodiment corresponding to the above method embodiment.", + "p0196": "FIG. 6 is a navigation device provided in an embodiment of the present application, which is applied to a server. The device includes:", + "p0197": "a first processing module 610 used to receive current position feature information sent by a mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and a database stored in the server, and take the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0198": "a determining module used to determine a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0199": "a second processing module used to determine a first route from the initial position to the target position, and display the first route on an electronic map of the mobile terminal.", + "p0200": "The embodiment of the present application provides a navigation device. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine the route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0201": "As an implementation of the present application, the determining module 620 is specifically used to display positions of respective empty areas on the electronic map of the mobile terminal based on the position information corresponding to the respective areas and the correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database, receive one of the positions of respective empty areas sent by the mobile terminal, and determine this position as the target position; or", + "p0202": "determine positions of all current empty areas based on position information corresponding to the respective areas and correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database, and determine a position of one of the empty areas as the target position.", + "p0203": "As an implementation of the present application, as shown in FIG. 7 , the device further includes:", + "p0204": "a detecting module 640 used to detect a current position of the mobile terminal, and display the current position on the electronic map of the mobile terminal.", + "p0205": "As an implementation of the present application, the detecting module 640 includes:", + "p0206": "an obtaining sub-module used to obtain a movement state of the mobile terminal based on an inertial sensor in the mobile terminal;", + "p0207": "a determining sub-module used to determine the current position of the mobile terminal based on the initial position of the mobile terminal and the movement state.", + "p0208": "As an implementation of the present application, the device further includes:", + "p0209": "a third processing module used to receive the current position feature information sent by the mobile terminal when receiving a position correction instruction sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database, and update the current position of the mobile terminal with this position.", + "p0210": "As an implementation of the present application, the detecting module 640 is specifically used to receive the current position feature information periodically sent by the mobile terminal, determine a position of an area matching the current position feature information based on the current position feature information and the database, and take the position as the current position.", + "p0211": "As an implementation of the present application, the detecting module 640 is also used to detect whether an offset between the current position and the first route is greater than a predetermined threshold. The device further includes:", + "p0212": "a fourth processing module used to determine a second route from the current position to the target position based on the current position and the target position, and display the second route on the electronic map of the mobile terminal, when the detecting module detects that the offset between the current position and the first route is greater than the predetermined threshold.", + "p0213": "As an implementation of the present application, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently; the current position feature information includes at least one of: identification information of a parking space, a license plate number or a comparison image.", + "p0214": "As an implementation of the present application, for each area, the feature image of the area is acquired by the image acquiring device corresponding to the area after detecting that content included in the area is changed.", + "p0215": "FIG. 8 is a navigation device provided in the present application, which is applied to a mobile terminal. The device includes:", + "p0216": "a first processing module 810 used to obtain current position feature information and send the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0217": "a second processing module 820 used to receive a route sent by the server and display the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on position information corresponding to the respective areas and correspondence relationship between identification information of the respective areas and the respective feature information currently stored in the database.", + "p0218": "The embodiment of the present application provides a navigation device. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0219": "As an implementation of the present application, as shown in FIG. 9 , the device further includes:", + "p0220": "a receiving module 830 used to receive a current position of the mobile terminal sent by the server and display the current position on the electronic map of the mobile terminal.", + "p0221": "As an implementation of the present application, the first processing module 810 is specifically used to receive current position feature information input by the user;", + "p0222": "acquire a feature image and take the feature image as the current position feature information; or", + "p0223": "acquire a feature image and extract the current position feature information in the feature image, wherein the current position feature information includes at least one of an image including a preset area in the feature image and a character in the preset area.", + "p0224": "As an implementation of the present application, the database stores the identification information and the position information corresponding to each of parking spaces and a license plate number or image of a vehicle parked currently; the current position feature information includes at least one of: identification information of a parking space, a license plate number or a comparison image.", + "p0225": "FIG. 10 is a navigation system provided by an embodiment of the present application, which includes image acquiring devices 110 installed in respective areas, a navigation device as shown in FIGS. 7 to 8 located in the server 120 , and a navigation device as shown in FIG. 9 located in the mobile terminal 130 .", + "p0226": "The embodiment of the present application provides a navigation system. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0227": "The embodiment of the present application provides a navigation system including image acquiring devices installed in respective areas, a server and a mobile terminal.", + "p0228": "The image acquiring devices are used to acquire feature images of corresponding areas;", + "p0229": "The server is used to perform the navigation method as described in FIGS. 1 to 3 ;", + "p0230": "The mobile terminal is used to perform the navigation method as described in FIGS. 4 to 5 .", + "p0231": "The embodiment of the present application provides a navigation system. The server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0232": "Correspondingly, the present application also provides a storage medium for storing executable program codes, which perform the navigation method described in the present application when being executed. The navigation method described in the present application includes:", + "p0233": "receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in a server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0234": "determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0235": "determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal.", + "p0236": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0237": "Correspondingly, the present application also provides a storage medium for storing executable program codes, which perform the navigation method described in the present application when being executing. The navigation method described in the present application includes:", + "p0238": "obtaining current position feature information, and sending the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and", + "p0239": "receiving a route sent by the server and displaying the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0240": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0241": "Correspondingly, the present application also provides an application, which is used to perform the navigation method described in the present application when being executed. The navigation method described in the present application includes:", + "p0242": "receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in a server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0243": "determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0244": "determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal.", + "p0245": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0246": "Correspondingly, the present application also provides an application, which is used to perform the navigation method described in the present application when being executed. The navigation method described in the present application includes:", + "p0247": "obtaining current position feature information, and sending the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and", + "p0248": "receiving a route sent by the server and displaying the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0249": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0250": "Correspondingly, the present application provides an electronic device comprising:", + "p0251": "a processor, a memory, a communication interface and a bus,", + "p0252": "wherein the processor, the memory and the communication interface are connected and communicated with each other via the bus;", + "p0253": "the memory stores executable program codes; and", + "p0254": "the processor performs the navigation method described in the present application by reading the executable program codes stored in the memory. The navigation method described in the present application includes:", + "p0255": "receiving current position feature information sent by a mobile terminal, determining a position of an area matching the current position feature information based on the current position feature information and a database stored in a server, and taking the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas;", + "p0256": "determining a position of one empty area of the areas as a target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database; and", + "p0257": "determining a first route from the initial position to the target position, and displaying the first route on an electronic map of the mobile terminal.", + "p0258": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0259": "Correspondingly, the present application provides an electronic device comprising:", + "p0260": "a processor, a memory, a communication interface and a bus,", + "p0261": "wherein the processor, the memory and the communication interface are connected and communicated with each other via the bus;", + "p0262": "the memory stores executable program codes; and", + "p0263": "the processor performs the navigation method described in the present application by reading the executable program codes stored in the memory. The navigation method described in the present application includes:", + "p0264": "obtaining current position feature information, and sending the current position feature information to a server, so that the server determines a position of an area matching the current position feature information based on the current position feature information and a database stored therein, and takes the position as an initial position, wherein the database is created by the server according to position information corresponding to respective areas and correspondence relationship between identification information of the respective areas and respective feature information, wherein the respective feature information is obtained based on feature images of respective areas and sent to the server, and the feature images of respective areas are acquired by image acquiring devices corresponding to the respective areas; and", + "p0265": "receiving a route sent by the server and displaying the route on an electronic map, wherein the route is a route from the initial position to a target position, which is determined by the server after determining a position of one empty area of the areas as the target position based on the position information corresponding to the respective areas and the correspondence relationship between the identification information of the respective areas and the respective feature information currently stored in the database.", + "p0266": "In the embodiment of the present application, the server can accurately determine the initial position based on the current position feature information sent by the mobile terminal and the database stored in the server, and can determine the position of one empty area in the database as the target position. Further, the server can determine a route from the initial position to the target position based on the initial position and the target position, and display the route on the electronic map of the mobile terminal. Therefore, it is possible to provide the user with accurate navigation information and make him find an empty parking space in a large parking lot quickly.", + "p0267": "For the device/system/storage medium/application program/electronic device embodiments, since they are substantially similar to the method embodiment, the descriptions thereof are relatively simple, and the related parts can refer to the part description of the method embodiment.", + "p0268": "It should be noted that the relationship terms herein such as \u201cfirst\u201d, \u201csecond\u201d and the like are only used to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply that there is any such actual relationship or sequence between these entities or operations. Moreover, the terms \u201ccomprise\u201d, \u201cinclude\u201d or any variants thereof are intended to cover a non-exclusive inclusion, such that processes, methods, articles or devices comprising a series of elements include not only those elements listed, but also other elements not specifically listed or the elements intrinsic to those processes, methods, articles, or devices. Without further limitations, elements limited by the statement \u201ccomprise a/an . . . \u201d do not exclude that there are additional identical elements in the processes, methods, articles, or devices which include the elements.", + "p0269": "All embodiments in the present description are described in a correlated manner, and identical or similar parts in various embodiments can be referred to one another. The description for each embodiment focuses on the differences with other embodiments. Especially, the device embodiments are described in a concise manner and related contents thereof can refer to the part description of the method embodiments, since the device embodiments are substantially similar to the method embodiments.", + "p0270": "Those ordinary skilled in the art should understand that all or part of the steps in the above method implementation can be implemented by a program to command the associated hardware, and the program can be stored in a computer readable storage medium, such as ROM/RAM, disk, CD and so on.", + "p0271": "Embodiments described above are just preferred embodiments of the present application, and not intended to limit the scope of the present application. Any modifications, equivalent substitutions, improvements or the like within the spirit and principle of the present application should be included in the scope of the present application.", + "features": { + "F1": "The method involves a server receiving current position feature information sent by a mobile terminal to determine an area's position matching the current position feature information using the server's database, which is created based on position information and the correspondence between area identification and feature information.", + "F2": "The method determines an initial position by using the position ascertained from the database as the starting point.", + "F3": "Feature images used to obtain respective feature information are acquired by image acquiring devices corresponding to the respective areas.", + "F4": "The method identifies a target position by determining the position of an empty area based on the area identification, feature information, and current details in the database.", + "F5": "A first route is determined from the initial position to the target position and displayed on an electronic map on the mobile terminal.", + "F6": "Positions of empty areas can be displayed on the electronic map, and one is selected as the target position based on user input from the mobile terminal or server-determined criteria.", + "F7": "The method includes detecting and displaying the current position of the mobile terminal on an electronic map after displaying the route.", + "F8": "Detecting the current position involves using the inertial sensor of the mobile terminal to obtain the movement state and determining the position based on the initial position and movement state.", + "F9": "Updating the current position involves receiving position correction instructions and current position feature information from the mobile terminal, then determining the matching area's position to update the terminal's position.", + "F10": "During periodic checks, the server receives and matches current position feature information, updating the current position accordingly.", + "F11": "If an offset between the current position and the route exceeds a predetermined threshold, a new route is determined and displayed.", + "F12": "The database stores identification information, position information for parking spaces, and parked vehicle details, incorporating them into current position feature information as needed.", + "F13": "Feature images of areas are acquired when a change in area content is detected.", + "F14": "The mobile terminal part of the navigation method involves obtaining and sending current position feature information to the server and displaying a server-determined route.", + "F15": "The navigation system consists of image acquiring devices, a server, and a mobile terminal, each fulfilling specific roles in the navigation process.", + "F16": "An electronic device comprises a processor, memory, communication interface, and a bus, functioning to execute the navigation method using stored program codes." + } + } + }, + { + "FAN": "79098180", + "CPC": [ + "A47K-005/12/05", + "A47K-005/12/17", + "A47K-007/04/6", + "A47K-2010/3226", + "B05B-011/108", + "B05B-012/00/4", + "E03D-009/08", + "E03D-2201/00", + "G01F-009/00/1", + "G01F-023/00/07", + "G06F-016/00", + "G06F-017/30", + "G06N-020/00", + "G06N-099/00/5", + "H04L-067/12" + ], + "Content": { + "title": "Universal dispenser monitor", + "pa01": "A retrofit dispenser monitor is disclosed. The dispenser monitor has a connector allowing it to be connected directly to one of a number of dispensers. The dispenser monitor also comprises a sensor configured to detect the dispensing action of the attached dispenser by learning the characteristics of the dispensing action. Information corresponding to the dispensing action is then transmitted to a remote location which allows the use of the dispenser to be monitored.", + "c-en-0001": "A retrofit dispenser monitor comprising: a connector configured to connect the dispenser monitor directly to a dispenser body of a dispenser such that there is physical contact between the dispenser monitor and the dispenser; a sensor configured to detect the attached dispensing action of the attached dispenser; a controller connected to the sensor, the controller being configured: to learn characteristics of a data signature which is transmitted through the body of the dispenser directly to the physically connected dispenser monitor and which corresponds to a dispensing action of the attached dispenser; and to recognize subsequent data signatures received from the sensor which corresponds to the learned characteristics in order to determine a measure of dispenser use; and a transmitter configured to transmit data relating to the determined measure of dispenser use.", + "c-en-0002": "The monitor according to claim 1 , wherein the connector is a releaseable connector.", + "c-en-0003": "The monitor according to claim 1 , wherein the connector comprises an adhesive area; a clamp or a magnet.", + "c-en-0004": "The monitor according to claim 1 , wherein the sensor comprises one or more of: a vibration sensor; a heat sensor; a light sensor; an electromagnetic interference sensor; a sound sensor; a volume sensor; a motion sensor; and an orientation sensor.", + "c-en-0005": "The monitor according to claim 1 , wherein the controller is configured to distinguish between a plurality of dispensing actions based on the received data signature.", + "c-en-0006": "The monitor according to claim 1 , wherein the controller is configured to predict characteristics of the next data signature based on the most recent recognized signatures.", + "c-en-0007": "The monitor according to claim 1 , wherein the controller is configured to determine whether the dispenser has been refilled based on received data.", + "c-en-0008": "The monitor according to claim 7 , wherein the controller is configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a refill action.", + "c-en-0009": "The monitor according to claim 8 , wherein the controller is configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a full dispenser.", + "c-en-0010": "The monitor according to claim 1 , wherein the controller is configured to determine whether the dispenser has been refilled based on a user interaction.", + "c-en-0011": "The monitor according to claim 1 , wherein the signature characteristics comprise one or more of: duration of dispensing; frequency of vibration during dispensing; and temperature profile during dispensing.", + "c-en-0012": "The monitor according to claim 1 , wherein the dispenser comprises one or more of: a soap dispenser; a sanitizer dispenser; a paper towel dispenser; a urinal; and a toilet.", + "c-en-0013": "The monitor according to claim 1 , wherein the dispenser monitor is configured to: compare a received signature with a library of signatures of known dispensers; and determine the type of the attached dispenser based on the comparison.", + "c-en-0014": "The monitor according to claim 1 , wherein the dispenser monitor comprises a universal asynchronous receiver/transmitter configured to allow the dispenser monitor to communicate with a remote device.", + "c-en-0015": "The monitor according to claim 1 , wherein the dispenser monitor comprises a magnetic sensor, and wherein the dispenser monitor is configured to suspend recording or detecting signatures while the magnetic sensor is detecting a particular magnetic field.", + "c-en-0016": "The monitor according to claim 1 , wherein the transmitter comprises one or more of: a wireless transmitter; a wired transmitter; a Bluetooth\u00ae transmitter; an IR transmitter; a cellular transmitter; a WiFi transmitter; and a radio-frequency transmitter.", + "c-en-0017": "A method, the method comprising: connecting a dispenser monitor directly to a dispenser body of a dispenser such that there is a physical contact between the dispenser monitor and the dispenser; detecting the attached dispensing action of the attached dispenser; learning characteristics of a data signature which is transmitted through the body of the dispenser directly to the physically connected dispenser monitor and which corresponds to a dispensing action; and recognizing subsequent data signatures which corresponds to the learned characteristics in order to determine a measure of dispenser use; and transmitting data relating to the determined measure of dispenser use.", + "c-en-0018": "A system comprising: a plurality of retrofit dispenser monitors according to claim 1 ; a remote electronic device having a receiver configured to receive data from each of the transmitters relating to the determined measure of dispenser use wherein the remote electronic device is configured to provide a notification when the dispensers require refilling.", + "c-en-0019": "The system of claim 18 , wherein the remote electronic device is a portable electronic device.", + "c-en-0020": "A retrofit dispenser monitor comprising: a connector configured to connect the dispenser monitor directly to a dispenser body of a dispenser such that there is physical contact between the dispenser monitor and the dispenser; a sensor configured to detect the attached dispensing action of the attached dispenser by measuring a signature which is transmitted through the body of the dispenser directly to the physically connected dispenser monitor; a controller connected to the sensor, the controller being configured to: compare received sensor data with a library of signatures of known dispensers; determine the type of the attached dispenser based on the comparison; and recognize subsequent data signatures received from the sensor which correspond to the determined dispenser type in order to determine a measure of dispenser use; and a transmitter configured to transmit data relating to the determined measure of dispenser use.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATION", + "p0002": "This application claims priority from U.S. provisional application No. 62/397,741 filed on Sep. 21, 2016 by Smart Wave Technologies Corp. This priority application is hereby incorporated by reference in its entirety.", + "p0003": "BACKGROUND", + "p0004": "Field of the Invention", + "p0005": "The invention relates to dispensers. In particular, the invention relates to dispenser monitors.", + "p0006": "Background", + "p0007": "Currently, if there are a number of dispensers and monitoring is desired then new dispensers generally have to be monitored with the monitoring technology built in. However, this is expensive and wasteful. It would be advantageous if existing dispenser could be used and their use monitored.", + "p0008": "U.S. Pat. No. 6,839,644 discloses a system for monitoring a plumbing system having a plurality of fixtures includes a sensor vibrationally coupled to the plumbing system and a processor. The sensor detects a vibration produced by the plumbing system, and generates a signal representative of the vibration. The processor receives the signal, compares the signal to a signal database that associates each of a plurality of stored signals with operation of one or more of the fixtures, and determines that the associated fixture or group of fixtures has been operated based on the comparison. The processor also stores a record of the operation of the associated fixture or group of fixtures.", + "p0009": "U.S. Pat. No. 7,405,653 discloses a residential monitoring system incorporates both physiological sensors and activity sensors. Some of the sensors can be in wireless communication with control circuitry.", + "p0010": "U.S. Pat. No. 7,825,813 discloses microprocessor technology used to detect routine sounds in a substantially enclosed environment to determine normal or abnormal activity or noise within the environment A device is situated in a substantially enclosed environment with audio input device similar to a microphone and an optional visual display for interaction with the local users. The device has the ability to be controlled locally via key-pad or USB port connected to a local laptop computer, or remotely via a phone line or Ethernet connection to internet. The device further performs audio pattern recognition using waveform matching scheme to detect the occurrence of pre-programmed sounds representing routine activities. The microprocessor counts the number occurrence of each recognizable sound for a particular interval and over a length of a day or other duration and reports to a remote server.", + "p0011": "U.S. Pat. No. 7,999,686 relates to an interactive packaging system for monitoring the usage of personal hygiene products, personal hygiene product packages, and restroom facilities. The interactive packaging system may include a plurality of usage monitoring devices each configured to detect physical stimulus associated with the usage of the product, package or facility with which the usage monitoring device is associated, and to transmit usage messages containing information relating to the detected physical stimulus. The interactive packaging system may further include a base station configured to receive the usage messages transmitted by the plurality of usage monitoring devices.", + "p0012": "U.S. Pat. No. 8,364,546 discloses a restroom monitoring system for monitoring attributes of fixtures within a restroom using sensors. Additional attributes are determined from the monitored attributes. Consumable usage levels are estimated based on predetermined consumption levels associated with usage states of the fixtures. The restroom monitoring system provides an indication of the need for replenishment of consumables based on the monitored attributes of the fixtures.", + "p0013": "U.S. Pat. No. 8,970,391 discloses toilet management systems and methods. In an example system, a central monitoring component compiles usage information from sensors stationed at multiple toilet units and compiles a report of usage patterns. The usage patterns are networked to potential users via a user interface available on a webpage or mobile communications device.", + "p0014": "US 2015/0178654 discloses methods and systems for managing an inventory. A database may store information regarding items owned by a user. The information regarding an item may include a quantity owned and one or more triggering events. Based on the occurrence of a triggering event, an order for the item may be placed without user intervention. Data to the database may be provided by one or more sensors. Triggering events may be defined in terms of sensor data. The triggering event may be defined by a user or through machine learning.", + "p0015": "US 2015/0207275 discloses techniques and systems are provided for providing power and/or communication to a device, such as a dispenser, a flush valve of a bathroom device, a hygiene opportunity measurement device, a hands free faucet, etc. For example, the device may send current device statuses, usage metrics, and/or other information to a remote source. The remote source may send instructions, data collection requests, functionality changes, and/or other information to the device.", + "p0016": "SUMMARY", + "p0017": "In accordance with the present disclosure, there is provided a retrofit dispenser monitor comprising:", + "p0018": "a connector configured to connect the dispenser monitor directly to a dispenser;", + "p0019": "a sensor configured to detect the attached dispensing action of the attached dispenser;", + "p0020": "a controller connected to the sensor, the controller being configured:", + "p0021": "- to learn characteristics of a data signature corresponding to a dispensing action of the attached dispenser; and - to recognize subsequent data signatures received from the sensor which correspond to the learned characteristics in order to determine a measure of dispenser use; and", + "p0022": "a transmitter configured to transmit data relating to the determined measure of dispenser use.", + "p0023": "The data signature may be considered to be a characteristic trace or sign that indicates the occurrence of a physical process or event. The controller may learn characteristics by identifying particular features of the received sensor data which correspond to a particular process or event .", + "p0024": "The measure of dispenser use may depend on the particular dispenser and/or product being dispensed. For example, a measure of dispenser use may correspond to the number of dispense actions . A measure of dispenser use may correspond to the quantity of product dispensed .", + "p0025": "The connector may be a releaseable connector. The connector may comprise an adhesive area; a clamp or a magnet. The connector may comprise a lock. It will be appreciated that other fasteners may be used. The connector may be configured not to require the dispenser to be modified .", + "p0026": "The sensor may comprise one or more of: a vibration sensor; a heat sensor; a light sensor; an electromagnetic interference sensor ; a sound sensor; a volume sensor; a motion sensor; and an orientation sensor.", + "p0027": "The controller may be configured to distinguish between a plurality of dispensing actions based on the received data signature.", + "p0028": "The controller may be configured to predict characteristics of the next data signature based on previous recognized signatures.", + "p0029": "The controller may be configured to determine whether the dispenser has been refilled based on received data.", + "p0030": "The controller may be configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a refill action.", + "p0031": "The controller may be configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a full dispenser.", + "p0032": "The controller may be configured to determine whether the dispenser has been refilled based on a user interaction.", + "p0033": "The signature characteristics may comprise one or more of: duration of dispensing; frequency of vibration during dispensing; and temperature profile during dispensing.", + "p0034": "The dispenser may comprise one or more of: a soap dispenser; a sanitizer dispenser; a paper towel dispenser; a urinal; and a toilet.", + "p0035": "The dispenser monitor may be configured to:", + "p0036": "compare a received signature with a library of signatures of known dispensers; and", + "p0037": "determine the type of the attached dispenser based on the comparison.", + "p0038": "The dispenser monitor may comprise a universal asynchronous receiver/transmitter configured to allow the dispenser monitor to communicate with a remote device. A remote device may comprise a processor, a receiver, a memory and computer program code. A remote device may also comprise a transmitter.", + "p0039": "The dispenser monitor may comprises a magnetic sensor, and the dispenser monitor may be configured not to record or detect signatures while the magnetic sensor is detecting a particular magnetic field.", + "p0040": "The transmitter may comprise one or more of: a wireless transmitter; a wired transmitter; a Bluetooth\u00ae transmitter; an IR transmitter; a cellular transmitter; a WiFi transmitter; and a radio-frequency transmitter.", + "p0041": "According to the present disclosure, there is also provided a system comprising a plurality of retrofit dispenser monitors as described herein, wherein each retrofit dispenser monitor is attached to a spatially separated dispenser and is configured to transmit use data to the same remote device. For example, one dispenser monitor may be attached to a urinal, whereas another dispenser monitor in the same system may be attached to a spatially separate paper towel dispenser.", + "p0042": "According to the present disclosure, there is also provided a system comprising:", + "p0043": "a plurality of retrofit dispenser monitors as described herein;", + "p0044": "a remote electronic device having a receiver configured to receive data from each of the transmitters relating to the determined measure of dispenser use", + "p0045": "wherein the remote electronic device is configured to provide a notification when the dispensers require refilling.", + "p0046": "The remote electronic device may be a portable electronic device.", + "p0047": "According to a further aspect, there is provided a retrofit dispenser monitor comprising:", + "p0048": "a connector configured to connect the dispenser monitor directly to a dispenser;", + "p0049": "a sensor configured to detect the attached dispensing action of the attached dispenser;", + "p0050": "a controller connected to the sensor, the controller being configured:", + "p0051": "- compare received sensor data with a library of signatures of known dispensers; and - determine the type of the attached dispenser based on the comparison; and - to recognize subsequent data signatures received from the sensor which correspond to the determined dispenser type in order to determine a measure of dispenser use; and", + "p0052": "a transmitter configured to transmit data relating to the determined measure of dispenser use.", + "p0053": "According to a further aspect, there is provided a method, the method comprising:", + "p0054": "connecting a dispenser monitor directly to a dispenser;", + "p0055": "detecting the attached dispensing action of the attached dispenser;", + "p0056": "learning characteristics of a data signature corresponding to a dispensing action; and", + "p0057": "recognizing subsequent data signatures which corresponds to the learned characteristics in order to determine a measure of dispenser use; and", + "p0058": "transmitting data relating to the determined measure of dispenser use.", + "p0059": "According to a further aspect, there is provided a method, the method comprising:", + "p0060": "- connecting a dispenser monitor directly to a dispenser; - detecting the attached dispensing action of the attached dispenser; - comparing received sensor data with a library of signatures of known dispensers; - determining the type of the attached dispenser based on the comparison; and - recognizing subsequent data signatures which correspond to the determined dispenser type in order to determine a measure of dispenser use; and - transmitting data relating to the determined measure of dispenser use.", + "p0061": "According to a further aspect, there is provided a computer program, the computer program comprising computer code configured, when run on a processor in a dispenser monitor connected directly to a dispenser:", + "p0062": "to enable detection of the attached dispensing action of the attached dispenser;", + "p0063": "to learn characteristics of a data signature corresponding to a dispensing action; and", + "p0064": "to recognize subsequent data signatures which corresponds to the learned characteristics in order to determine a measure of dispenser use; and", + "p0065": "to enable transmission of data relating to the determined measure of dispenser use.", + "p0066": "According to a further aspect, there is provided a computer program, the computer program comprising computer code configured, when run on a processor in a dispenser monitor connected directly to a dispenser:", + "p0067": "to enable detection of the attached dispensing action of the attached dispenser;", + "p0068": "to compare received sensor data with a library of signatures of known dispensers;", + "p0069": "to determine the type of the attached dispenser based on the comparison; and", + "p0070": "to recognize subsequent data signatures which correspond to the determined dispenser type in order to determine a measure of dispenser use; and", + "p0071": "to transmit data relating to the determined measure of dispenser use.", + "p0072": "The connector may enable a direct physical connection between the dispenser and the attached dispenser monitor.", + "p0073": "A dispenser may be considered to be a container, package, device, or vending machine for holding and dispensing a product in small amounts. The dispenser may be configured to dispense product in response to a user interaction. The dispenser may be configured to allow a plurality of dispenses before the available product is depleted. The product may be dispensed in discrete units such as paper cups or paper towels. The product may be dispensed as a continuous flow, such as sugar, water or coffee.", + "p0074": "A dispenser may also encompass devices configured to consume a portion or fractional part of a stored product in response to a user interaction. For example, a urinal may dispense water and consume deodorizer in response to the user flushing. Or a camping stove may be configured to consume gas in response to being turned on. A dispenser may also encompass devices configured to distribute product from a remote or central supply. For example, a water dispenser may not have an integral container but be plumbed into the mains to supply portions of water to a user.", + "p0075": "A retrofit dispenser monitor may be a dispenser monitor capable of being retrofitted to a dispenser to monitor dispensing actions of that dispenser.", + "p0076": "A controller may comprise a processor . The controller may comprise memory . The controller may be configured to run computer program code configured to allow the controller to learn or recognize dispense signatures. The computer program code may be stored on a non-transitory medium such as a CD or DVD.", + "p0077": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0078": "Various objects, features and advantages of the invention will be apparent from the following description of particular embodiments of the invention, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of various embodiments of the invention. Similar reference numerals indicate similar components.", + "p0079": "FIG. 1 a is a front view of a coffee dispenser with a first embodiment of a retrofit dispenser monitor.", + "p0080": "FIGS. 1 b and 1 c are front and rear perspective views of the retrofit dispenser monitor of FIG. 1 a.", + "p0081": "FIG. 1 d is a schematic of the components of the dispenser monitor.", + "p0082": "FIG. 1 e is a front view of the coffee dispenser of FIG. 1 a in use.", + "p0083": "FIG. 1 f is a front view of a water dispenser in conjunction with the dispenser monitor of FIG. 1 a.", + "p0084": "FIG. 2 a is a front view of a soap dispenser in use with a second embodiment of a retrofit dispenser monitor.", + "p0085": "FIG. 2 b is a graph showing how the frequency of the dispense actions change as the soap is depleted from the soap dispenser of FIG. 2 a.", + "p0086": "FIG. 2 c is a front view of the soap dispenser of FIG. 2 a during refilling.", + "p0087": "FIG. 3 is a schematic of a plurality of dispensers with associated dispenser monitors transmitting data to a remote device.", + "p0088": "DETAILED DESCRIPTION", + "p0089": "With reference to the figures, a retrofit dispenser monitor is disclosed. The dispenser monitor has a connector allowing it to be connected directly to one of a number of dispensers. The dispenser monitor also comprises a sensor configured to detect the dispensing action of the attached dispenser by learning the characteristics of the dispensing action. Information corresponding to the dispensing action is then transmitted on to a remote location which allows the use of the dispenser to be monitored.", + "p0090": "In the context of this invention, dispensers are typically used in a variety of settings, and perform a number of roles. In each case, a dispenser is generally an automatic machine or container that is designed to release a specific amount of something. That is, an advantage of using a dispenser is that it helps reduce the amount of manpower required to distribute goods to individuals. However, generally dispensers are provided with a finite quantity of product to dispense, and the time taken to dispense this quantity depends on the frequency of use by the end users of the dispenser.", + "p0091": "This means that such dispensers must be monitored in some way to ensure that the product for dispensing is replenished when stocks get low. One way of doing this is manually sending a person to inspect product levels at regular intervals. This increases the manpower required to maintain the dispensers in operation. This is particularly important where there may be a large number of dispensers .", + "p0092": "Another option is to use dispensers which have built-in monitors. These monitors may comprise sensors designed to measure the quantity of product remaining in the dispenser and be configured to provide an alert when the dispenser needs refilled. However, this sort of dispenser monitor is integrally linked to the particular dispenser and cannot be readily retrofitted to other dispensers. Such devices cannot be used in existing dispensers systems without replacing the dispensers themselves.", + "p0093": "Therefore, there is a need for a retrofit dispenser which can be used with a variety of different pre-existing dispensers to allow the monitoring of those dispensers whilst mitigating the need for manual inspection.", + "p0094": "Various aspects of the invention will now be described with reference to the figures. For the purposes of illustration, components depicted in the figures are not necessarily drawn to scale. Instead, emphasis is placed on highlighting the various contributions of the components to the functionality of various aspects of the invention. A number of possible alternative features are introduced during the course of this description. It is to be understood that, according to the knowledge and judgment of persons skilled in the art, such alternative features may be substituted in various combinations to arrive at different embodiments of the present invention.", + "p0095": "FIG. 1 a shows an embodiment of a retrofit dispenser monitor 100 connected to a dispenser 150 . In this case, the dispenser is a pump coffee dispenser comprising an insulated vessel 151 , a pump having a plunger 153 , and an outlet 152 . When the user depresses the plunger 153 , the dispenser 150 is configured to dispense a portion of coffee from within the vessel 151 through the outlet 152 .", + "p0096": "In this case, a retrofit dispenser monitor 100 has been connected to the surface of the dispenser.", + "p0097": "FIGS. 1 b and 1 c are close-up views of the dispenser monitor 100 . FIG. 1 b shows an obverse perspective view of the dispenser monitor 100 . FIG. 1 b shows the connector 101 which is configured to connect the dispenser monitor 100 directly to the dispenser. In this case, the connector 101 comprises a ring of adhesive which allows the dispenser monitor to be stuck to a surface of the dispenser. The connector 101 may have a deformable shape in order for the connector surface to connect securely to dispenser surfaces with different shapes. In this case, the sensor 102 is located within the ring connector 101 to allow physical contact between the sensor 102 and the dispenser. In this case, the sensor is a vibration sensor. It will be appreciated that as vibrations travel through a medium, the sensor can be housed within a body of the dispenser monitor and still be able to detect the vibrations which are transmitted through the body of the dispenser body.", + "p0098": "Other embodiments may comprise other sensors such as one or more of: a tilt sensor; a vibration sensor; a shock sensor; an accelerometer; and a magnetic sensor.", + "p0099": "A magnetic sensor may allow an engineer to stop and start the main function of the monitor without actually touching the device and therefore collecting only relevant information not distorted by the manipulations during connecting the monitor to UART interface, attaching the monitor to the dispenser, etc. That is, the dispenser monitor may be configured not to record or detect signatures in the while the magnetic sensor is detecting a particular magnetic field .", + "p0100": "FIG. 1 c is a reverse perspective view of the dispenser monitor 100 . On this side, the dispenser monitor comprises a transmitter 104 configured to transmit data relating to the determined measure of dispenser use. It will be appreciated the dispenser may also comprise a power source such as a solar panel or battery .", + "p0101": "FIG. 1 d is a schematic of the components of the dispenser monitor module. In this case, the dispenser monitor module comprises:", + "p0102": "a connector 101 configured to connect the dispenser monitor directly to a dispenser;", + "p0103": "a sensor 102 configured to detect the attached dispensing action of the attached dispenser;", + "p0104": "a controller 103 connected to the sensor, the controller being configured", + "p0105": "- to learn characteristics of a data signature corresponding to a dispensing action of the attached dispenser; and - to recognize subsequent data signatures received from the sensor which corresponds to the learned characteristics in order to determine a measure of dispenser use; and", + "p0106": "a transmitter 104 configured to transmit data relating to the determined measure of dispenser use.", + "p0107": "In this case, the controller 103 comprises a processor 111 and a memory 112 having computer program code 113 installed. In this case, the controller is a low power microcontroller with RAM.", + "p0108": "In order to learn the characteristics of the data signature corresponding to a dispensing action of the attached dispenser, in this case, the controller 103 is switched into a learning mode. In this case, the controller 103 can be controlled remotely through the transmitter 104 acting as a receiver for commands wirelessly. It will be appreciated that, in other embodiments, the dispenser monitor 100 can be controlled directly using a user interface on the dispenser monitor itself .", + "p0109": "When the controller 103 is in learning mode, the dispenser monitor is configured to detect vibration signatures and associate those vibration signatures with dispensing actions. For example, the dispenser monitor 100 may prompt the user to dispense a number of times . The controller will look for three vibration signatures which have similar characteristics. The characteristics may include, for example: duration, average vibration frequency, and/or vibration frequency profile.", + "p0110": "In this case, the controller 103 is then configured to generate a characteristics envelope so that if a signature is received which is within the characteristics envelope, it will be recognized and identified as a dispensing action.", + "p0111": "The parameters delimiting the characteristics envelope may be based on the variation in the dispensing actions performed in the learning mode. The range of the characteristics envelope may be related to the variance in the vibrations signatures detected during a learning mode. For example, if the duration of the dispensing actions during the learning mode were 1 second; 2 seconds and 1.5 seconds, the controller may determine that the characteristics envelope should encompass vibration signatures of between 0.5 seconds and 3 seconds. However, if the duration of the dispensing actions during the learning mode were 1 second; 1.1 seconds and 1.3 seconds, the controller may determine that the characteristics envelope should encompass vibration signatures of between 0.8 seconds and 1.5 seconds. This may allow the dispenser monitor to reduce the number of false positives where the signatures corresponding to a dispensing action are more consistent.", + "p0112": "It will be appreciated that because the dispenser monitor 100 learns the vibration signal, the position of the dispenser monitor is not crucial to the operation of the dispenser monitor. That is, although the vibration signature of a dispensing action may be different depending on whether, for example, the dispenser monitor is attached to the top or bottom of the dispenser, any differences will be accommodated when the device learns the particular dispense signature corresponding to the attachment position.", + "p0113": "The controller 103 is then, in this case, configured to switch from learning mode to an operational mode. The switch from learning mode to operational mode may be based on one or more of the following: a predetermined time threshold being exceeded ; the requisite number of dispensing actions being detected; and the user providing a switch command to the controller.", + "p0114": "In this case, when in the operational mode, the controller is configured to be in low power sleep mode until one of the sensors is tripped by a user action . That is this embodiment is configured such that the sensor wakes up the controller from its sleep mode.", + "p0115": "Throughout the event the controller stores in its memory the signals coming from all the sensors to collect the signature of the event. By analyzing the signature the controller is able to differentiate the event as a dispense or maintenance event or as a should-be ignored mechanical disturbance caused by noise, people traffic, etc.", + "p0116": "When in the operational mode, the dispenser monitor 100 is configured to monitor data received from the sensor 102 and to recognize subsequent data signatures received from the sensor which corresponds to the learned characteristics in order to determine a measure of dispenser use. For example, the controller may be configured to detect the number of times the dispenser has been activated to dispense, in this case, a cup of coffee 159 . That is, each time a dispenser signature is detected may be associated with a dispense event. The dispense event may be associated with a fixed volume of product . It will be appreciated that in other cases, the dispense event and the measure of dispenser use may be associated with a quantity of product based on the received dispense signature. For example, a recognized signature with a long duration may be associated with a larger quantity than a dispense signature with a short duration. This association may take place at the controller or at a remote device configured to receive data from the transmitter of the dispenser monitor.", + "p0117": "The controller 103 is configured to control the transmitter 104 to transmit data relating to the determined measure of dispenser use. In this case, the dispenser monitor is configured to transmit an indication to a remote server periodically. That is, there may be a predetermined schedule for transmitting dispenser data . This may reduce the energy consumption of the transmitter. This allows a remote user to monitor how the dispenser is being used.", + "p0118": "In this case, the dispenser monitor comprises a UART interface configured to allow the controller to communicate with a remote device to, for example, upload the collected event information or download new signature data, programming information, new firmware, etc. It will be appreciated that other embodiments may allow the dispenser to be monitored in real time .", + "p0119": "Other embodiments may not use real-time monitoring. For example, the dispenser monitor may transmit data to a nearby \u201cremote server\u201d that's within range via Bluetooth . When the service person arrives, the \u201cremote server\u201d may connect with their cell phone and use the connection to send the data collected to the cloud or to a central server. Such non-real-time systems mentioned may reduce hardware and implementation costs.", + "p0120": "In other embodiments, the dispenser monitor 100 may be configured to store data relating to use and to transmit data upon request from the user. In other embodiments, the dispenser monitor may be provided with a quantity value corresponding to the number of dispensing actions or amount of product which can be taken before the product which is being dispensed will be exhausted. For example, in a coffee dispenser, the dispenser monitor may be provided with the volume of coffee in the dispenser and the volume of coffee dispensed in each dispensing action. In this case, the dispenser monitor may be configured to send a notification to the remote electronic device indicating that the volume remaining in the dispenser is 10% .", + "p0121": "In this case, the connector 101 is a releasable connector allowing the dispenser monitor module to be removed from one dispenser 150 and re-used with another dispenser. In this case, the user can simply pull the dispenser monitor away from the coffee dispenser and reattach the dispenser monitor to, for example, a water dispenser 170 as shown in FIG. 1 f . The releasable connector may be configured such that when removed, the dispenser is unchanged from its state before the dispenser was attached .", + "p0122": "When the dispenser monitor has been reattached, the dispenser monitor can learn the dispense signature corresponding to the water dispenser by being put into the learning mode again. Once this is done, the dispenser monitor can monitor water dispenses rather than coffee dispenses.", + "p0123": "In the water dispenser case, there are two outlets 172 a , 172 b , each of which can be activated for various durations in order to dispense more or less water. Each outlet 172 a , 172 b may correspond to a different dispense signature . Multiple dispense signatures may also occur where a dispenser is configured to perform multiple functions . In this case, the dispenser monitor 100 is configured to recognize water being dispensed from either outlet as a dispense event. This may be enabled by using both outlets 172 a , 172 b during the learning phase. For example, if four dispenses were requested, the user could perform two dispenses with one outlet 172 a and two dispenses with the other outlet 172 b . In response, the dispenser monitor 100 would create a characteristics envelope which encompasses dispensing through either outlet 172 a , 172 b.", + "p0124": "It will be appreciated that, in some embodiments, it may be possible to distinguish between different functions . For example, the controller may be configured to receive information relating to a plurality of functions which the device can perform. Then a learning phase would be performed for each of the various functions which would associate a particular characteristics envelope for each function . In cases where the multiple characteristics envelopes did not overlap, any received dispense signature could be associated with the corresponding dispense function.", + "p0125": "In this case, the dispense signature could be limited to a subset of the detectable parameters . The duration of the vibration in this case may then be used as an indication of the volume of water dispensed. For example, the duration of the vibration may be proportional to the volume of water dispensed. This data can then be stored in the controller or transmitted to a remote electronic device by the transmitter to give an indication of the volume of water consumed.", + "p0126": "Important elements which allow the retrofit dispenser monitor 100 to function as a \u2018universal\u2019 dispenser monitor include that the connector 101 is suitable for connection to a variety of devices. In this case, the dispenser monitor can be connected to any surface of the dispenser. Another feature which is important in this regard is that the dispenser monitor 100 is configured to learn the signature from the dispenser 150 , 170 . This means that the location of the dispenser monitor 100 is not critical because any variations in dispenser signature due to the variation in position of the dispenser monitor will be accounted for because the dispenser monitor will simply learn the appropriate dispense signature corresponding to its current location. Another important factor is that the sensor is configured to sense features of the dispense event which are distributed . For example, in contrast with detecting the level of product remaining , the present dispenser monitor can detect the dispense events from a location on the dispenser removed from the dispense mechanism or dispenser container.", + "p0127": "It will be appreciated that some embodiments may be pre-configured to recognize known devices, such a soap dispenser, toilet, urinal, paper towel dispensers.", + "p0128": "For example, knowing the devices the dispenser monitor would be retrofitted on, the dispenser monitor may come pre-programmed with a \u201clibrary\u201d of signatures of known dispensers. This library would be created as part of the development efforts and would allow the dispenser monitor to figure out what it was attached to by comparing the signature of the dispenser it is attached to with the signatures in its library.", + "p0129": "That is, some embodiments of the dispenser monitor may not have to \u201clearn\u201d because the dispenser monitor has already been \u201ctaught\u201d or pre-programmed. This is a \u201cstick on\u201d and walk away type device. This device may require less effort on the installer's part .", + "p0130": "FIG. 2 a shows a second embodiment 200 of a dispenser monitor comprising:", + "p0131": "a connector configured to connect the dispenser monitor 200 directly to a dispenser 250 ;", + "p0132": "a sensor configured to detect the attached dispensing action of the attached dispenser;", + "p0133": "a controller connected to the sensor, the controller being configured", + "p0134": "- to learn characteristics of a data signature corresponding to a dispensing action of the attached dispenser; and - to recognize subsequent data signatures received from the sensor which corresponds to the learned characteristics in order to determine a measure of dispenser use; and", + "p0135": "a transmitter 204 configured to transmit data relating to the determined measure of dispenser use.", + "p0136": "In this case, the controller comprises a BLE module. The BLE module, in this example, is a dual function module that performs the functions of the low power microcontroller and RF radio to communicate with a mobile phone app and BLE enabled gateway", + "p0137": "In this case, the dispenser monitor is attached to a soap dispenser which can be activated by the user placing their hand 290 below a remote sensor 255 . Using a retrofit dispenser monitor may be particularly useful in dispensers with remote activation because the dispense signatures in such dispensers may be more consistent as they are not affected by contact with the user.", + "p0138": "The dispenser in this case comprises a container 251 which holds a quantity of soap 260 for dispensation in portions. In this case, the dispensed portions of soap have a predetermined and consistent volume.", + "p0139": "However, in this case, because the mass of the soap 260 is a significant proportion of the mass of the dispenser 250 , as the stored soap 260 diminishes, the characteristics of the vibration may change. For example the frequency of vibration may increase towards the end of the stored soap 291 .", + "p0140": "In this case, the dispenser monitor 200 is configured to record changes in the dispense signatures over time to more accurately detect future dispense signatures. That is, the dispenser monitor is simultaneously learning and detecting.", + "p0141": "In this case, the dispenser monitor is configured to monitor the variation of characteristics of the dispense signatures as a function of number of dispenses and identify trends in the dispense signatures. For example, as shown in FIG. 2 b , in this case, the vibrational frequency of the last 10 dispenses of this dispenser has been steadily increasing as the soap level goes down. In this case, the controller is configured to adjust the characteristics window by a frequency interval corresponding to the average increase in frequency for each of the last 10 dispenses. In FIG. 2 b , the vibration characteristics window for the 10 th dispense is shown in solid horizontal lines. After the 10 th dispense has been detected, the vibrational characteristics window is adjusted accordingly for the 11 th dispense and is shown in FIG. 2 c as dotted horizontal lines. This may help improve accuracy and reduce the incidences of false positives by using a narrower characteristics window for each dispense. That is, the characteristics window would need to be much wider to be able to detect all of the dispense vibrational signatures shown in FIG. 2 b and so this wider characteristics window may incorrectly recognize incidental vibrations as corresponding to dispense actions.", + "p0142": "It will be appreciated that other effects may cause a change in the dispense signature from the dispenser .", + "p0143": "As well as allowing the apparatus to more accurately recognize a dispense signature, changes in the vibrational signature may also allow the dispenser monitor to monitor the status of the device. For example, the dispense signature may be used to determine whether the battery in the dispenser needs to be replaced.", + "p0144": "In certain embodiments , refilling the dispenser may result in a step change in the dispense signature. The apparatus may be configured to recognize a refill in order to change the characteristics window to correspond to a full dispenser.", + "p0145": "In this case, refilling the dispenser involves removing the empty container as shown in FIG. 2 c , and inserting a new full container .", + "p0146": "In this case, the dispenser monitor 200 is configured to look for signatures corresponding to two distinct characteristics windows. The first characteristics window corresponds to the predicted next dispense based on previous dispenses as described above, and the second characteristics window corresponds to the window around the expected signature corresponding to a full dispenser . When a new dispense signature falls outside the predicted characteristics window but within the full dispenser characteristics window, the dispense monitor is configured to recognize that the dispenser has been refilled. In response, the predicted characteristics window will be reset and determined based on the signatures received since the refill.", + "p0147": "Another approach would be to have a reset button on the dispenser monitor which would be activated by the user each time the dispenser is refilled. In response to activation of the reset button, the dispenser button could prompt the user to initiate a learning protocol or reset to a previously learned characteristics window corresponding to a fill dispenser.", + "p0148": "Other embodiments may be configured to recognize the refill step. For example, the dispenser may be configured to recognize the vibration signature of the dispenser being refilled by learning the characteristic signature of a refilling action in an analogous way to how the dispense action is learned. For example, the dispenser monitors may be configured to recognize a large amplitude vibration as corresponding to a refill step. Other dispenser monitors may be configured to be located within the housing of the dispenser and recognize an increase in light levels as being associated with a refill. Other dispenser monitors may be configured to be located on the refill container and be configured to detect movement of the container itself and/or being detached from the container and reattached to the refill contains as corresponding to a refill. Some dispenser monitor may use a combination of these approaches.", + "p0149": "In other embodiments, the dispenser monitor may comprise multiple different sensors. For example, a dispenser monitor may comprise a vibration sensor and a heat sensor. Using multiple sensors may increase accuracy and help distinguish between different dispense events. For example, in a coffee machine, making and dispensing a coffee shot may be recognized by detecting a characteristic vibration and heat signature. That is, the heat signature may be a good discriminator between a coffee shot dispensing event and a refill event.", + "p0150": "It will be appreciated that multiple dispensers may transmit data to the same central hub . This may allow the replenishing of supplies and servicing of the multiple dispensers to be coordinated. For example, in a public washroom environment, the various dispensers may have been used to different degrees. This would allow the washroom attendant to take the appropriate supplied when going to refill the washroom dispensers.", + "p0151": "This is shown in FIG. 3 which is a schematic of an embodiment of a system comprising a plurality of retrofit dispenser monitors 300 a - e ; and a remote electronic device 398 having a receiver configured to receive data from each of the transmitters relating to the determined measure of dispenser use/", + "p0152": "In this case, each dispenser monitor is attached to a respective dispenser 350 a - b , 370 a - c . The dispensers in this case comprise two urinals 350 a - b and three paper towel dispensers 370 a - c.", + "p0153": "The remote electronic device in this case is a portable electronic device such as a mobile phone or tablet computer. These can be carried by the washroom attendant and, in this case, is configured to wirelessly connect with each of the dispenser monitors within a washroom to allow transmission of dispense action data. The data can be displayed on a display of the remote device to allow the user to review the status of each of the dispensers. In this case, the remote device is configured to display the proportion of product remaining.", + "p0154": "The portable electronic device in this case is configured to forward this data on to the cloud 399 . It will be appreciated that other systems may be configured to allow sharing of data between computers or servers. The remote device may be used to control each of the dispenser monitors. For example, the remote device may be used to switch the dispenser monitor to a learning mode or to reset the dispenser monitor after a refill or battery change.", + "p0155": "In other embodiments, there may be a local remote device nearby to a plurality of dispensers to collate and aggregate data from the dispensers within range. For example, there may be a local \u201cremote device\u201d in the washroom that talks to all the washroom dispensers. In this case, instead of the service person connecting to each dispenser with his mobile phone or other portable device, he only connects to the local \u201cremove device\u201d, which has all the information collected by the dispenser monitors.", + "p0156": "That is, there are at least three ways to access data collected by the dispenser monitors:", + "p0157": "- Dispenser monitors each connect directly to service person's portable electronic device . The portable electronic device may then send data to the cloud. - Dispenser monitors each connect to local \u201cremote device\u201d in washroom. Service person's electronic device then connects to local \u201cremote device\u201d . The portable electronic device may then send data to the cloud. - Dispenser monitors connects to local \u201cremote device\u201d in washroom. The local \u201cRemote device\u201d may then send data directly to the cloud.", + "p0158": "It will be appreciated that this list is not exhaustive as there are potentially more ways to get data to the cloud .", + "p0159": "In other embodiments, the remote electronic device may be configured to provide a notification when the dispensers require refilling .", + "p0160": "It will be appreciated that data received by the remote electronic device may be aggregated over time and processed to generate reports for end-users.", + "p0161": "Although the present invention has been described and illustrated with respect to preferred embodiments and preferred uses thereof, it is not to be so limited since modifications and changes can be made therein which are within the full, intended scope of the invention as understood by those skilled in the art.", + "features": { + "F1": "A retrofit dispenser monitor comprises a connector configured to connect the dispenser monitor directly to a dispenser body of a dispenser such that there is physical contact between the dispenser monitor and the dispenser.", + "F2": "A sensor is configured to detect the attached dispensing action of the attached dispenser.", + "F3": "A controller is connected to the sensor, configured to learn characteristics of a data signature, which is transmitted through the body of the dispenser directly to the physically connected dispenser monitor and corresponds to a dispensing action.", + "F4": "The controller is configured to recognize subsequent data signatures received from the sensor that correspond to the learned characteristics in order to determine a measure of dispenser use.", + "F5": "A transmitter is configured to transmit data relating to the determined measure of dispenser use.", + "F6": "The connector can be a releaseable connector.", + "F7": "The connector comprises an adhesive area, a clamp, or a magnet.", + "F8": "The sensor comprises one or more of a vibration sensor, a heat sensor, a light sensor, an electromagnetic interference sensor, a sound sensor, a volume sensor, a motion sensor, and an orientation sensor.", + "F9": "The controller is configured to distinguish between a plurality of dispensing actions based on the received data signature.", + "F10": "The controller is configured to predict characteristics of the next data signature based on the most recent recognized signatures.", + "F11": "The controller is configured to determine whether the dispenser has been refilled based on received data.", + "F12": "The controller is configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a refill action.", + "F13": "The controller is configured to determine whether the dispenser has been refilled based on detecting a data signature corresponding to a full dispenser.", + "F14": "The controller is configured to determine whether the dispenser has been refilled based on a user interaction.", + "F15": "The signature characteristics comprise one or more of duration of dispensing, frequency of vibration during dispensing, and temperature profile during dispensing.", + "F16": "The dispenser comprises one or more of a soap dispenser, a sanitizer dispenser, a paper towel dispenser, a urinal, and a toilet.", + "F17": "The dispenser monitor is configured to compare a received signature with a library of signatures of known dispensers and determine the type of the attached dispenser based on the comparison.", + "F18": "The dispenser monitor comprises a universal asynchronous receiver/transmitter configured to allow communication with a remote device.", + "F19": "The dispenser monitor comprises a magnetic sensor and is configured to suspend recording or detecting signatures while the magnetic sensor is detecting a particular magnetic field.", + "F20": "The transmitter comprises one or more of a wireless transmitter, a wired transmitter, a Bluetooth\u00ae transmitter, an IR transmitter, a cellular transmitter, a WiFi transmitter, and a radio-frequency transmitter.", + "F21": "A method comprises connecting a dispenser monitor directly to a dispenser body of a dispenser such that there is physical contact, detecting the attached dispensing action, learning characteristics of a data signature, recognizing subsequent data signatures, and transmitting data relating to dispenser use.", + "F22": "A system comprises a plurality of retrofit dispenser monitors and a remote electronic device having a receiver configured to receive data from each of the transmitters and provide a notification when the dispensers require refilling.", + "F23": "The remote electronic device in the system is a portable electronic device.", + "F24": "A retrofit dispenser monitor includes a sensor that measures a signature transmitted through the body of the dispenser, and the controller compares received sensor data with a library of signatures of known dispensers to determine the type and measure dispenser use." + } + } + }, + { + "FAN": "85685768", + "CPC": [ + "G06F-009/46", + "G06F-009/46/6", + "G06F-009/48", + "G06F-009/48/81", + "G06F-009/50", + "G06F-009/50/16", + "G06F-009/54", + "G06F-009/54/6", + "G06F-016/00", + "G06F-016/18", + "G06F-016/1815", + "G06F-016/23", + "G06F-016/2365", + "G06F-016/242", + "G06F-016/2438", + "G06F-016/27", + "G06F-2209/5018", + "G06F-2209/548", + "Y02D-010/00" + ], + "Content": { + "title": "Distributed database cluster system, data synchronization method and storage medium", + "pa01": "The present disclosure provides a distributed database cluster system, a data synchronization method, and a storage medium. The distributed database cluster system includes: a first synchronization clock module, a first data synchronization module and a plurality of first database groups, each first database group of the plurality of first database groups includes a first master database, and the first synchronization clock module is configured to periodically send a clock synchronization instruction; the first data synchronization module is configured to respectively encapsulate database operation instructions, which are recorded by each first master database between two adjacent clock synchronization instructions sent by the first synchronization clock module, into a first operation instruction block, and send the first operation instruction block. By means of the present disclosure, the problem in the related art that data synchronized by different data nodes cannot achieve global time consistency is solved, so that data records synchronized by different data nodes are based on the same time reference point, thereby ensuring global consistency in the recorded data.", + "c-en-0001": "A distributed database cluster system, comprising: a first synchronization clock module , a first data synchronization module and a plurality of first database groups , wherein each first database group of the plurality of first database groups comprises a first master database, the first synchronization clock module is configured to periodically send a clock synchronization instruction to each first master database ; and the first data synchronization module is configured to respectively encapsulate database operation instructions, which are recorded by each first master database between two adjacent clock synchronization instructions sent by the first synchronization clock module , into a first operation instruction block, and send the first operation instruction block , wherein the first synchronization clock module is configured to periodically send the clock synchronization instruction to each first master database; the first master database is configured to write received database operation instructions into a log file of the first master database, and write the received clock synchronization instruction into the log file; and the first data synchronization module is configured to respectively acquire the log file of each first master database, encapsulate the database operation instructions located between two adjacent clock synchronization instructions in the log file into the first operation instruction block, and send the first operation instruction block, wherein the distributed database cluster system further comprises: a first consistency filtering module , wherein the first data synchronization module is configured to send the first operation instruction block to the first consistency filtering module ; and the first consistency filtering module is configured to receive the first operation instruction block sent by the first data synchronization module , perform global consistency filtering on the first operation instruction block, encapsulate database operation instructions, which satisfy a filtering condition, in the first operation instruction block into a second operation instruction block, and send the second operation instruction block to a distributed database cluster system at a peer end, wherein the filtering condition comprises that the database operation instructions are in a global transaction submitted state; or, the filtering condition comprises that the database operation instructions are in the global transaction submitted state and that the database operation instructions comply with a data object filtering rule corresponding to the distributed database cluster system at the peer end; wherein when the distributed database cluster system further comprises the first consistency filtering module , the first consistency filtering module is further configured to: buffer database operation instructions, which are in a global transaction unsubmitted state, in the first operation instruction block, and when a next first operation instruction block corresponding to a current first master database sent by the first data synchronization module is received, add the buffered database operation instructions to the first operation instruction block and then perform global consistency filtering on the first operation instruction block, wherein the global transaction submitted state refers to the instructions that have been executed, the global transaction unsubmitted state refers to the instructions that have not been executed.", + "c-en-0002": "The distributed database cluster system as claimed in claim 1, wherein the first data synchronization module is configured to send the first operation instruction block to a distributed database cluster system at a peer end;", + "c-en-0003": "The distributed database cluster system as claimed in claim 1, wherein when the distributed database cluster system further comprises the first consistency filtering module , the distributed database cluster system further comprises: a global transaction management module , wherein the global transaction management module is configured to manage a global transaction identifier capable of uniquely identifying a global transaction, and record a state and a life cycle of each global transaction; the first synchronization clock module is configured to request a current active transaction list of each first master database periodically, record the current active transaction list and a timestamp for acquiring the current active transaction list in a synchronization table corresponding to the first master database, and put the timestamp and information of the current active transaction list in the clock synchronization instruction sent to the first master database; and the first consistency filtering module is configured to search for a latest current active transaction list recorded in the synchronization table corresponding to a current first master database, remove database operation instructions corresponding to global transaction IDs in the latest current active transaction list, and encapsulate remaining database operation instructions into the second operation instruction block as the database operation instructions, which satisfy the filtering condition, in the global transaction submitted state.", + "c-en-0004": "The distributed database cluster system as claimed in claim 1, wherein the first data synchronization module is further configured to: receive synchronization position request information sent from a distributed database cluster system at a peer end, wherein the synchronization position request information carries a synchronization position between the distributed database cluster system and the distributed database cluster system at the peer end; and encapsulate, according to the synchronization position carried in the synchronization position request information, the database operation instructions recorded by each first master database.", + "c-en-0005": "The distributed database cluster system as claimed in any one of claims 1 to 4, wherein the first data synchronization module is further configured to: receive an operation instruction block sent by a distributed database cluster system at a peer end, wherein the operation instruction block comprises a first operation instruction block or a second operation instruction block; and execute database operation instructions in the received operation instruction block, wherein preferably, the first data synchronization module is configured to execute the database operation instructions in the received operation instruction block in the following manner: persisting the database operation instructions in the received operation instruction block into a playback file; and parsing the playback file, splitting the database operation instructions in the playback file by lines, and putting the database operation instructions into different working threads for processing, wherein preferably, the first data synchronization module is configured to parse the playback file, split the database operation instructions in the playback file by lines, and put the database operation instructions into different working threads for processing in the following manner: splitting the database operation instructions in the received operation instruction block into different data lines according to table names and primary keys, wherein database operation instructions in a same data line compose a transaction according to a sequence; and putting database operation instructions of different transactions into different working threads for processing.", + "c-en-0006": "The distributed database cluster system as claimed in claim 5, wherein the first data synchronization module is further configured to: persist a position where the database operation instructions are executed successfully into a file after the database operation instructions in the received operation instruction block are executed successfully.", + "c-en-0007": "The distributed database cluster system as claimed in claim 5, wherein in a case where the distributed database cluster system is a backup distributed database cluster system for the distributed database cluster system at the peer end, the first data synchronization module is configured to invoke the first consistency filtering module when the distributed database cluster system needs to become a master distributed database cluster system in place of the distributed database cluster system at the peer end; the first consistency filtering module is configured to: acquire a synchronization table corresponding to each first master database, determine a timestamp of a latest synchronization for each first master database according to the synchronization table, and determine an earliest timestamp from the timestamp of the latest synchronization for each first master database as a switching reference time point; for each first master database for which the timestamp of the latest synchronization is later than the switching reference time point, roll data of the first master database back to a recorded state at the switching reference time point, and restore data of the first master database to a globally consistent state, wherein preferably, the first consistency filtering module is configured to roll the data of the first master database back to the recorded state at the switching reference time point, and restore the data of the first master database to the globally consistent state in the following manner: generating a reverse database operation instruction for each database operation instruction after the switching reference time point, and executing the reverse database operation instruction; and generating a reverse database operation instruction for each database operation instruction which is in a global transaction unsubmitted state at the switching reference time point, and executing the reverse database operation instruction.", + "c-en-0008": "A distributed database cluster system, comprising: a second data synchronization module and a plurality of second database groups , wherein each second database group of the plurality of second database groups comprises a second master database, the second data synchronization module is configured to receive an operation instruction block sent by a distributed database cluster system at a peer end, wherein the operation instruction block comprises a first operation instruction block, database operation instructions, which are recorded by each first master database of the distributed database cluster system at the peer end between two adjacent clock synchronization instructions, are encapsulated in the first operation instruction block, wherein the filtering condition comprises that the database operation instructions are in a global transaction submitted state; and execute database operation instructions in the received operation instruction block, wherein the distributed database cluster system comprises a distributed database cluster system as claimed in any one of claims 1 to 7.", + "c-en-0009": "The distributed database cluster system as claimed in claim 8, wherein the second data synchronization module is configured to execute the database operation instructions in the received operation instruction block in the following manner: persisting the database operation instructions in the received operation instruction block into a playback file; and parsing the playback file, splitting the database operation instructions in the playback file by lines, and putting the database operation instructions into different working threads for processing, wherein preferably, the second data synchronization module is configured to parse the playback file, split the database operation instructions in the playback file by lines, and put the database operation instructions into different working threads for processing in the following manner: splitting the database operation instructions in the received operation instruction block into different data lines according to table names and primary keys, wherein database operation instructions in a same data line compose a transaction according to a sequence; and putting database operation instructions of different transactions into different working threads for processing.", + "c-en-0010": "The distributed database cluster system as claimed in any one of claims 8 to 9, wherein the second data synchronization module is further configured to: persist a position where the database operation instructions are executed successfully into a file after the database operation instructions in the received operation instruction block are executed successfully.", + "c-en-0011": "The distributed database cluster system as claimed in any one of claims 8 to 9, further comprising a second consistency filtering module , wherein the second data synchronization module is configured to invoke the second consistency filtering module when the distributed database cluster system needs to become a master distributed database cluster system in place of the distributed database cluster system at the peer end; the second consistency filtering module is configured to: acquire a synchronization table corresponding to each second master database, determine a timestamp of a latest synchronization for each second master database according to the synchronization table, and determine an earliest timestamp from the timestamp of the latest synchronization for each second master database as a switching reference time point; for each second master database for which the timestamp of the latest synchronization is later than the switching reference time point, roll data of the second master database back to a recorded state at the switching reference time point, and restore the data of the second master database to a globally consistent state, wherein preferably, the second consistency filtering module is configured to roll the data of the second master database back to the recorded state at the switching reference time point, and restore the data of the second master database to a globally consistent state in the following manner: generating a reverse database operation instruction for each database operation instruction after the switching reference time point, and executing the reverse database operation instruction; and generating a reverse database operation instruction for each database operation instruction which is in a global transaction unsubmitted state at the switching reference time point, and executing the reverse database operation instruction.", + "c-en-0012": "A data synchronization method, applied to a first distributed database cluster system claimed in any one of claims 1 to 7, wherein each first database group of the plurality of first database groups comprises a first master database, the distributed database cluster system further comprising a first consistency filtering module , the method comprising: periodically sending, by the first synchronization clock module , a clock synchronization instruction to each first master database ; writing, by the first master database, the received database operation instructions into a log file of the first master database, and writing the received clock synchronization instruction into the log file; acquiring, by the first data synchronization module , respectively the log file of each first master database, encapsulating the database operation instructions located between two adjacent clock synchronization instructions in the log file into the first operation instruction block, and sending the first operation instruction block to the first consistency filtering module ; receiving, by the first consistency filtering module , the first operation instruction block sent by the first data synchronization module, performing global consistency filtering on the first operation instruction block, encapsulating database operation instructions, which satisfy a filtering condition, in the first operation instruction block into a second operation instruction block, and sending the second operation instruction block to a distributed database cluster system at a peer end, wherein the filtering condition comprises that the database operation instructions are in a global transaction submitted state; or, the filtering condition comprises that the database operation instructions are in the global transaction submitted state and that the database operation instructions comply with a data object filtering rule corresponding to the distributed database cluster system at the peer end and buffering, by the first consistency filtering module , database operation instructions, which are in a global transaction unsubmitted state, in the first operation instruction block, and when a next first operation instruction block corresponding to a current first master database sent by the first data synchronization module is received, adding the buffered database operation instructions to the first operation instruction block and then perform global consistency filtering on the first operation instruction block, wherein the global transaction submitted state refers to the instructions that have been executed, the global transaction unsubmitted state refers to the instructions that have not been executed.", + "c-en-0013": "A data synchronization method, applied to a second distributed database cluster system claimed in any one of claims 8 to 11, wherein each second database group of the plurality of second database groups comprises a second master database, the method comprising: receiving, by the second distributed database cluster system, an operation instruction block sent by a first distributed database cluster system, wherein the operation instruction block comprises a first operation instruction block or a second operation instruction block, database operation instructions, which are recorded by each first master database of the distributed database cluster system at the peer end between two adjacent clock synchronization instructions, are encapsulated in the first operation instruction block, and database operation instructions, which satisfy a filtering condition, in the database operation instructions recorded by each first master database of the distributed database cluster system at the peer end between two adjacent clock synchronization instructions are encapsulated in the second operation instruction block, wherein the filtering condition comprises that the database operation instructions are in a global transaction submitted state; and executing, by the second distributed database cluster system, database operation instructions in the received operation instruction block.", + "p0001": "Technical Field", + "p0002": "The present disclosure relates to the field of distributed databases, and in particular, to a distributed database cluster system, a data synchronization method, and a storage medium.", + "p0003": "Background", + "p0004": "In recent years, more and more organizations begin to focus on and use open source database products, such as MySQL . With the development of services and technologies, database technologies are developing towards clustering. The data synchronization technology between clusters is a fundamental technology of disaster recovery, and research in the development of efficient and consistent data synchronization methods between two data centers is of high and deep significance.", + "p0005": "Currently, in the field of distributed databases, data synchronization between data clusters is mainly implemented by point-to-point data migration. However, due to various factors, such as differences in servers and networks, data synchronized by different data nodes between the clusters cannot achieve global time consistency. For example, when two data nodes A and B in the master cluster synchronize their respective data to two data nodes A' and B' in the backup cluster, at a certain moment, A' may synchronize the data of A at the moment T0, but B' may synchronize the data of B at the moment T1. If the switching occurs at this time, the service needs to be migrated to the backup cluster, and since the data of the backup cluster is not globally consistent, the service cannot be switched to the backup cluster immediately. Furthermore, because there is no reference time point or clock, the service data cannot be recovered to a consistent time point.", + "p0006": "In addition to switching scenarios between master and backup database clusters, in other scenarios involving data synchronization, including but not limited to incremental data subscription and incremental data extraction, there is also a problem that data synchronized by different data nodes between clusters cannot achieve global time consistency.", + "p0007": "US 2007/239950 A1 discloses a method, system, and article of manufacture, wherein a primary control unit sends a selected time to a plurality of secondary control units. US6516327B1 discloses a system, method, and processor readable medium containing code for causing a processor to synchronize a secondary database system, such as one stored on a palmtop electronic device, with a distributed database system, comprising a main database and at least one replica database.", + "p0008": "Summary", + "p0009": "Distributed database cluster systems-and data synchronization methods according to the invention are defined in the appended set of claims.", + "p0010": "Brief Description of the Drawings", + "p0011": "The accompanying drawings described herein are used to provide a deeper understanding of the embodiments of the present disclosure and form a part of the present disclosure, and the schematic embodiments of the present disclosure and the description thereof are used to explain the embodiments of the present disclosure, and do not limit the embodiments of the present disclosure. In the figures:", + "p0012": "Fig. 1 is a schematic diagram showing the architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure; Fig. 2 is a schematic diagram showing a first exemplary architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure; Fig. 3 is a schematic diagram showing a second exemplary architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure; Fig. 4 is a schematic diagram showing the architecture of a distributed database cluster system according to Embodiment 2 of the present disclosure; Fig. 5 is a schematic diagram showing an exemplary architecture of a distributed database cluster system according to Embodiment 2 of the present disclosure; Fig. 6 is a flowchart of a data synchronization method according to Embodiment 3 of the present disclosure; Fig. 7 is a flowchart of a data synchronization method according to Embodiment 4 of the present disclosure; Fig. 8 is a schematic diagram showing the architecture of a distributed database data synchronization apparatus for ensuring global consistency according to Embodiment 5 of the present disclosure; Fig. 9 is a detailed working flowchart of a data synchronization method according to Embodiment 5 of the present disclosure; Fig. 10 is a schematic diagram of splitting and encapsulating of SQL statements according to Embodiment 6 of the present disclosure; Fig. 11 is a flowchart of parallel processing of SQL statement lines according to Embodiment 7 of the present disclosure; Fig. 12 is a flowchart of real-time global consistency filtering according to Embodiment 8 of the present disclosure; Fig. 13 is a flowchart of final global consistency filtering according to Embodiment 8 of the present disclosure; Fig. 14 is a schematic diagram showing the system architecture for data synchronization of a distributed database cluster under a high performance mode according to Embodiment 9 of the present disclosure; Fig. 15 is a schematic diagram showing the system architecture for data synchronization of a distributed database cluster under a high performance mode according to Embodiment 10 of the present disclosure; Fig. 16 is a schematic diagram showing the system architecture of a distributed database quasi-real-time backup according to Embodiment 11 of the present disclosure; and Fig. 17 is a schematic diagram showing the system architecture of data extraction based on distributed database incremental data according to Embodiment 12 of the present disclosure.", + "p0013": "Description of embodiments", + "p0014": "A distributed database cluster system is composed of a plurality of database groups. Each database group is composed of a plurality of independent database nodes, data of database nodes in a database group is completely consistent, and only one database node in the database group provides a writing service to the outside device, and other data nodes synchronize the data to the local through a replication technology between the databases. A database function is provided to an application in a data service manner, and a unified standard interface such as an SQL interface is provided to the application.", + "p0015": "For synchronization between distributed database cluster systems, for example, data synchronization between data clusters, incremental data subscription, incremental data extraction, etc., data synchronization schemes in the related art often have problems and defects in the incapability of providing global consistency.", + "p0016": "In view of the above problems, the embodiments of the present disclosure provide a distributed database cluster system, a data synchronization method, and a storage medium. Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings in conjunction with embodiments. It should be noted that the embodiments and the features in the embodiments of the present application can be combined with each other without conflict.", + "p0017": "It should be noted that the terms \"first\", \"second\" and the like in the description and claims of the present disclosure and the figures described above are used to distinguish similar objects, and are not used to describe a specific order or a precedence order.", + "p0018": "Embodiment 1", + "p0019": "This embodiment provides a distributed database cluster system. Fig. 1 is a schematic diagram showing the architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure. As shown in Fig. 1 , the distributed database cluster system may include a first synchronization clock module 12, a first data synchronization module 14, and a plurality of first database groups 16, where each first database group 16 of the plurality of first database groups 16 includes a first master database .", + "p0020": "The first synchronization clock module 12 is configured to periodically send a clock synchronization instruction.", + "p0021": "The first data synchronization module 14 is configured to respectively encapsulate database operation instructions, which are recorded by each first master database between two adjacent clock synchronization instructions sent by the first synchronization clock module, into a first operation instruction block, and send the first operation instruction block.", + "p0022": "Through the above solution, the first synchronization clock module 12 in the distributed database cluster system periodically sends a clock synchronization instruction. In a data synchronization process, the first data synchronization module 14 encapsulates database operation instructions recorded by each master database between two adjacent clock synchronization instructions into a first operation instruction block for sending, so that for each master database, the data packets serving as the basis for data synchronization are ensured to have the same time reference point. Thus, the problem in the related art that data synchronized by different data nodes cannot achieve global time consistency can be solved. The data records for synchronization of different data nodes are ensured to be based on the same time reference point, thereby guaranteeing global consistency in terms of data records.", + "p0023": "In the distributed database cluster system, there may be many manners for the first data synchronization module 14 to determine which instructions are the database operation instructions recorded by each first master database between the two adjacent clock synchronization instructions sent by the first synchronization clock module 12, so that the first data synchronization module 14 can encapsulate the database operation instructions into the first operation instruction block. An implementation that is easy to implement and does not need to greatly change the existing processing mode of the distributed database cluster system is provided as follows.", + "p0024": "As an exemplary embodiment, the first synchronization clock module 12 is configured to periodically send the clock synchronization instruction to each first master database; the first master database is configured to write received database operation instructions into a log file of the first master database, and write the received clock synchronization instruction into the log file; the first data synchronization module 14 is configured to respectively acquire the log file of each first master database, encapsulate the database operation instructions located between two clock synchronization instructions in the log file into the first operation instruction block, and send the first operation instruction block. In this way, the master database writes the received clock synchronization instructions and the common database operation instructions into the log file according to a normal processing manner. The first data synchronization module 14 may read the log file and segment the database operation instructions according to the clock synchronization instructions in the log file, and encapsulate the database operation instructions located between two clock synchronization instructions into the first operation instruction block. In order to facilitate the log recording of the master database, a relatively common database operation statement may be adopted to generate a clock synchronization instruction, so that the master database can conveniently write the clock synchronization instruction into the log file along with common database operation statements. For an existing distributed database cluster system, a preferred database operation statement is SQL, that is, a synchronous clock SQL statement may be used as the clock synchronization instruction, and the database operation instructions may also be SQL statements.", + "p0025": "As described above, the solution in this embodiment performs the following operations for operation instructions recorded by each master database: the database operation instructions recorded between two adjacent clock synchronization instructions are encapsulated into a first operation instruction block for sending. Thus, it is ensured that for each master database, the data packets based on which the data synchronization is performed all have the same time reference point, so that the operation instruction recording plane in the database reaches global consistency. In a case where the processing task distribution in the database cluster is relatively balanced or the processing resource bandwidth is sufficient, the execution duration of each operation instruction is very small and the operation instructions are under a relatively uniform distribution. In this case, global consistency can substantially be achieved at the execution layer of the database. In a practical distributed database cluster system, there may occasionally be a case where the processing task distribution in the database cluster is less unbalanced or the processing resource bandwidth is small. In this case, there may be differences in the execution states of different processing tasks in the same database cluster. In this case, at least one of the following two processing manners may be adopted to ensure global consistency of the database in the execution layers.", + "p0026": "Final global consistency scheme", + "p0027": "In this solution, the first data synchronization module 14 is configured to send the first operation instruction block to a distributed database cluster system at a peer end. In this case, the first operation instruction block sent to the distributed database cluster system of the peer end contains database operation instructions recorded in the master database between two synchronization clock signals, and no consistency filtering is performed for these database operation instructions. The data synchronization module of the distributed database cluster system at the peer end normally synchronizes these database operation instructions to the corresponding master database of the distributed database cluster system at the peer end. However, the distributed database cluster system at the peer end performs global consistency filtering when cluster switching is performed for the service. This embodiment is suitable for scenarios where the disaster recovery system has high requirements on RPO indexes, since in this solution data can be efficiently transmitted to a peer cluster and persisted, thereby ensuring that the RPO is as small as possible. A specific implementation of performing a global consistency filtering at the accepter end of data synchronization when cluster switching is performed for the service in this processing mode may refer to Embodiment 8.", + "p0028": "Real time global consistency scheme", + "p0029": "According to the real-time global consistency scheme, a first consistency filtering module 22 is added in the distributed database cluster system. Fig. 2 is a schematic diagram showing a first exemplary architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure. As shown in Fig. 2 , the distributed database cluster system further includes a first consistency filtering module 22.", + "p0030": "The first data synchronization module 14 is configured to send the first operation instruction block to the first consistency filtering module 22. The first consistency filtering module 22 is configured to receive the first operation instruction block sent by the first data synchronization module 14, perform global consistency filtering on the first operation instruction block, encapsulate database operation instructions, which satisfy a filtering condition, in the first operation instruction block into a second operation instruction block, and send the second operation instruction block to a distributed database cluster system at a peer end, wherein the filtering condition includes that the database operation instructions are in a global transaction submitted state. Through this scheme, the database operation instructions that have not been executed may be filtered from the first operation instruction block, and only the database operation instructions that have been executed are reserved and encapsulated into a second operation instruction block and sent to the second distributed database cluster system. In this way, the distributed database cluster system at the peer end synchronizes data that has been executed by the current distributed database cluster system, so that the data synchronized to the backup cluster is always in a consistent state. This embodiment is suitable for scenarios where the disaster backup system has high requirements on RTO indexes. The service may take over the peer distributed database cluster at any time, and the RTO may be equal to zero. A specific implementation of performing global consistency filtering in real time at the synchronization initiator in this processing mode may refer to Embodiment 8.", + "p0031": "Considering that in the scenarios of incremental data subscription and incremental data extraction, the distributed database cluster system at the peer end has a filtering requirement for received data and only data objects satisfying a specific condition need to be sent to the distributed database cluster system at the peer end, this filtering task can also be executed by the first consistency filtering module 22. A filtering condition may be added, for example, the filtering condition may further include that the database operation instructions comply with a data object filtering rule corresponding to the distributed database cluster system at the peer end.", + "p0032": "For database operation instructions that have not been executed , the first consistency filtering module 22 is further configured to buffer the database operation instructions, which are in a global transaction unsubmitted state, in the first operation instruction block, and when a next first operation instruction block corresponding to the current first master database sent by the first data synchronization module 14 is received, add the buffered database operation instructions to the first operation instruction block and then perform global consistency filtering on the first operation instruction block.", + "p0033": "In order to perform global transaction management inside the distributed database cluster system, so that the first consistency filtering module 22 can conveniently determine whether a database operation instruction is currently in a global submitted state or not, some improvement can be made to the distributed database cluster system. Fig. 3 is a schematic diagram showing a second exemplary architecture of a distributed database cluster system according to Embodiment 1 of the present disclosure. As shown in Fig. 3 , the distributed database cluster system further includes a global transaction management module 32, and the global transaction management module 32 is configured to manage a global transaction identifier capable of uniquely identifying a global transaction, and record a state and a life cycle of each global transaction.", + "p0034": "The first synchronization clock module 12 is configured to request a current active transaction list of each first master database periodically, record the current active transaction list and a timestamp for acquiring the current active transaction list in a synchronization table corresponding to the first master database, and put the timestamp and information of the current active transaction list in the clock synchronization instruction sent to the first master database. The first consistency filtering module 22 is configured to search for a latest current active transaction list recorded in the synchronization table corresponding to a current first master database, removing database operation instructions corresponding to global transaction IDs in the latest current active transaction list, and encapsulate remaining database operation instructions into the second operation instruction block as the database operation instructions, which satisfy the filtering condition, in the global transaction submitted state.", + "p0035": "As an exemplary implementation, before data synchronization processing is performed, a synchronization position between a distributed database cluster system at a peer end and a current distributed database cluster system may be acquired. In this case, the first data synchronization module 14 is further configured to receive synchronization position request information sent from a distributed database cluster system at the peer end, wherein the synchronization position request information carries a synchronization position between the distributed database cluster system and the distributed database cluster system at the peer end; and encapsulate, according to the synchronization position carried in the synchronization position request information, the database operation instructions recorded by each first master database.", + "p0036": "Described above are the functions performed by various modules of the distributed database cluster system at the initiator end of data synchronization. It should be noted that, in practical applications, the role of the distributed database cluster system often changes, for example, the distributed database cluster system may currently be a master distributed database cluster system, and needs to be used as an initiator of data synchronization. However, after the distributed database cluster system becomes faulty, the distributed database cluster system may become a backup distributed database cluster system. In this case, the distributed database cluster system serves as an acceptor of data synchronization. Therefore, as an exemplary implementation, various modules of the distributed database cluster system also have the following functions.", + "p0037": "As an accepter end of data synchronization, after receiving a data synchronization packet sent by an initiator end, the distributed database cluster system needs to execute contents of the data synchronization packet to complete data synchronization. In this case, the first data synchronization module 14 is further configured to receive an operation instruction block sent by a distributed database cluster system at a peer end, and execute database operation instructions in the received operation instruction block.", + "p0038": "As an exemplary embodiment, the first data synchronization module 14 is configured to execute the database operation instructions in the received operation instruction block in the following manner:", + "p0039": "persisting the database operation instructions in the received operation instruction block into a playback file; and parsing the playback file, splitting the database operation instructions in the playback file by lines, and putting the database operation instructions into different working threads for processing.", + "p0040": "As an exemplary implementation, the first data synchronization module 14 is configured to parse the playback file, split the database operation instructions in the playback file by lines, and put the database operation instructions into different working threads for processing in the following manner:", + "p0041": "splitting the database operation instructions in the received operation instruction block into different data lines according to table names and primary keys, wherein database operation instructions in a same data line compose a transaction according to a sequence; and putting database operation instructions of different transactions into different working threads for processing.", + "p0042": "In order to get prepared for the task restoration resulted from playback failure, as an exemplary implementation, the first data synchronization module 14 may be further configured to persist a position where the database operation instructions are executed successfully into a file after the database operation instructions in the received operation instruction block are executed successfully.", + "p0043": "As described in the final global consistency scheme, the distributed database cluster system at the peer end performs global consistency filtering when cluster switching is performed for the service. Considering a case where the distributed database cluster system serves as a backup distributed database cluster system for the distributed database cluster system at the peer end, the first data synchronization module 14 may invoke the first consistency filtering module 22 when the distributed database cluster system needs to become a master distributed database cluster system in place of the distributed database cluster system at the peer end.", + "p0044": "In the exemplary embodiment, the first consistency filtering module 22 is configured to:", + "p0045": "acquire a synchronization table corresponding to each first master database, determine a timestamp of a latest synchronization for each first master database according to the synchronization table, and determine an earliest timestamp from the timestamp of the latest synchronization for each first master database as a switching reference time point; for each first master database for which the timestamp of the latest synchronization is later than the switching reference time point, roll data of the first master database back to a recorded state at the switching reference time point, and restore data of the first master database to a globally consistent state.", + "p0046": "As an exemplary implementation, the first consistency filtering module 22 is configured to roll the data of the first master database back to the recorded state at the switching reference time point, and restore the data of the first master database to the globally consistent state in the following manner:", + "p0047": "generating a reverse database operation instruction for each database operation instruction after the switching reference time point, and executing the reverse database operation instruction; and generating a reverse database operation instruction for each database operation instruction which is in a global transaction unsubmitted state at the switching reference time point, and executing the reverse database operation instruction.", + "p0048": "Embodiment 2", + "p0049": "This embodiment provides another distributed database cluster system. The distributed database cluster system may be used as an accepter end of data synchronization. Certainly, in a case where the distributed database cluster system is provided with a function of an initiator end of data synchronization at the same time, the distributed database cluster system may also be used as an initiator end of data synchronization, and in this case reference may be made to Embodiment 1. Fig. 4 is a schematic diagram showing the architecture of a distributed database cluster system according to Embodiment 2 of the present disclosure. As shown in Fig. 4 , the distributed database cluster system may include a second data synchronization module 42 and a plurality of second database groups 44, where each second database group 44 of the plurality of second database groups 44 includes a second master database .", + "p0050": "The second data synchronization module 42 is configured to receive an operation instruction block sent by a distributed database cluster system at a peer end. The operation instruction block includes a first operation instruction block or a second operation instruction block. Database operation instructions, which are recorded by each first master database of the distributed database cluster system at the peer end between two adjacent clock synchronization instructions, are encapsulated in the first operation instruction block. Database operation instructions, which satisfy a filtering condition, in the database operation instructions recorded by each first master database of the distributed database cluster system at the peer end between two adjacent clock synchronization instructions are encapsulated in the second operation instruction block. The filtering condition includes that the database operation instructions are in a global transaction submitted state.", + "p0051": "The second data synchronization module 42 is further configured to execute database operation instructions in the received operation instruction block.", + "p0052": "Through the above solution, the distributed database cluster system serving as the accepter end of data synchronization may execute, by virtue of the second data synchronization module 42, the operation instruction block sent by the distributed database cluster system at the initiator end of data synchronization, thereby completing the data synchronization process. Since for database operation instructions recorded by each master database in the first distributed database cluster system, the operation instruction block carries recorded database operation instructions between two adjacent clock synchronization instructions, so that for each master database, the data packets serving as the basis for data synchronization are ensured to have the same time reference point. Thus, the problem in the related art that data synchronized by different data nodes cannot achieve global time consistency can be solved. The data records for synchronization of different data nodes are ensured to be based on the same time reference point, thereby guaranteeing global consistency in terms of data records.", + "p0053": "As an exemplary embodiment, the second data synchronization module 42 is configured to execute the database operation instructions in the received operation instruction block in the following manner:", + "p0054": "persisting the database operation instructions in the received operation instruction block into a playback file; and parsing the playback file, splitting the database operation instructions in the playback file by lines, and putting the database operation instructions into different working threads for processing.", + "p0055": "As an exemplary implementation, the second data synchronization module 42 is configured to parse the playback file, split the database operation instructions in the playback file by lines, and put the database operation instructions into different working threads for processing in the following manner:", + "p0056": "splitting the database operation instructions in the received operation instruction block into different data lines according to table names and primary keys, wherein database operation instructions in a same data line compose a transaction according to a sequence; and putting database operation instructions of different transactions into different working threads for processing.", + "p0057": "In order to get prepared for the task restoration resulted from playback failure, as an exemplary implementation, the second data synchronization module 42 may be further configured to persist a position where the database operation instructions are executed successfully into a file after the database operation instructions in the received operation instruction block are executed successfully.", + "p0058": "Fig. 5 is a schematic diagram showing an exemplary architecture of a distributed database cluster system according to Embodiment 2 of the present disclosure. For the final global consistency scheme, the distributed database cluster system may perform global consistency filtering when cluster switching is performed for the service. Considering that the distributed database cluster system may serve as a backup distributed database cluster system, as shown in Fig. 5 , the distributed database cluster system may further include a second consistency filtering module 52.", + "p0059": "The second data synchronization module 42 is configured to invoke the second consistency filtering module 52 when the distributed database cluster system needs to become a master distributed database cluster system in place of the distributed database cluster system at the peer end.", + "p0060": "The second consistency filtering module 52 is configured to:", + "p0061": "acquire a synchronization table corresponding to each second master database, determine a timestamp of a latest synchronization for each second master database according to the synchronization table, and determine an earliest timestamp from the timestamp of the latest synchronization for each second master database as a switching reference time point; for each second master database for which the timestamp of the latest synchronization is later than the switching reference time point, roll data of the second master database back to a recorded state at the switching reference time point, and restore the data of the second master database to a globally consistent state.", + "p0062": "As an exemplary implementation, the second consistency filtering module 52 is configured to roll the data of the second master database back to the recorded state at the switching reference time point, and restore the data of the second master database to a globally consistent state in the following manner:", + "p0063": "generating a reverse database operation instruction for each database operation instruction after the switching reference time point, and executing the reverse database operation instruction; and generating a reverse database operation instruction for each database operation instruction which is in a global transaction unsubmitted state at the switching reference time point, and executing the reverse database operation instruction.", + "p0064": "Embodiment 3", + "p0065": "This embodiment describes a processing flow at an initiator end of data synchronization. Fig. 6 is a flowchart of a data synchronization method according to Embodiment 3 of the present disclosure. The data synchronization method is applied to a first distributed database cluster system including a plurality of first database groups 16, wherein each first database group 16 of the plurality of first database groups 16 includes a first master database. As shown in Fig. 6 , the method includes the following operations S602 and S604.", + "p0066": "In S602, the first distributed database cluster system periodically sends a clock synchronization instruction.", + "p0067": "In S604, the first distributed database cluster system respectively encapsulates database operation instructions, which are recorded by each first master database between two adjacent clock synchronization instructions, into a first operation instruction block, and sends the first operation instruction block.", + "p0068": "Through the above solution, the first distributed database cluster system periodically sends a clock synchronization instruction. During data synchronization, for the operation instructions recorded in each master database of the first distributed database cluster system, the database operation instructions recorded between two adjacent clock synchronization instructions are encapsulated into the first operation instruction block for sending. Thus, it is ensured that for each master database, the data packets based on which the data synchronization is performed all have the same time reference point. Thus, the problem in the related art that data synchronized by different data nodes cannot achieve global time consistency can be solved. The data records for synchronization of different data nodes are ensured to be based on the same time reference point, thereby guaranteeing global consistency in terms of data records.", + "p0069": "There may be many manners to determine which instructions are the database operation instructions recorded by each first master database between the two adjacent clock synchronization instructions, so that the database operation instructions can be encapsulated into the first operation instruction block. An implementation that is easy to implement and does not need to greatly change the existing processing mode of the distributed database cluster system is provided as follows.", + "p0070": "As an exemplary embodiment, the first distributed database cluster system periodically sends the clock synchronization instruction to each first master database. Before the first distributed database cluster system respectively encapsulates the database operation instructions recorded by each first master database between two adjacent clock synchronization instructions into a first operation instruction block, the method further includes: the first master database writes received database operation instructions into a log file of the first master database, and writes the received clock synchronization instruction to the log file. The operation that the first distributed database cluster system encapsulates the database operation instructions recorded by each first master database between two adjacent clock synchronization instructions into a first operation instruction block includes: the first distributed database cluster system respectively acquires the log file of each first master database, and encapsulates the database operation instructions located between two clock synchronization instructions in the log file into the first operation instruction block. In this way, the master database writes the received clock synchronization instructions and the common database operation instructions into the log file according to a normal processing manner. The first distributed database cluster system may read the log file and segment the database operation instructions according to the clock synchronization instructions in the log file, and encapsulate the database operation instructions located between two clock synchronization instructions into the first operation instruction block. In order to facilitate the log recording of the master database, a relatively common database operation statement may be adopted to generate a clock synchronization instruction, so that the master database can conveniently write the clock synchronization instruction into the log file along with common database operation statements. For an existing distributed database cluster system, a preferred database operation statement is SQL, that is, a synchronous clock SQL statement may be used as the clock synchronization instruction, and the database operation instructions may also be SQL statements.", + "p0071": "As described above, the solution in this embodiment performs the following operations for operation instructions recorded by each master database: the database operation instructions recorded between two adjacent clock synchronization instructions are encapsulated into a first operation instruction block for sending. Thus, it is ensured that for each master database, the data packets based on which the data synchronization is performed all have the same time reference point, so that the operation instruction recording plane in the database reaches global consistency. In a case where the processing task distribution in the database cluster is relatively balanced or the processing resource bandwidth is sufficient, the execution duration of each operation instruction is very small and the operation instructions are under a relatively uniform distribution. In this case, global consistency can substantially be achieved at the execution layer of the database. In a practical distributed database cluster system, there may occasionally be a case where the processing task distribution in the database cluster is less unbalanced or the processing resource bandwidth is small. In this case, there may be differences in the execution states of different processing tasks in the same database cluster. In this case, at least one of the following two processing manners may be adopted to ensure global consistency of the database in the execution layers.", + "p0072": "Final global consistency scheme", + "p0073": "In this solution, the first distributed database cluster system sends the first operation instruction block to the second distributed database cluster system. In this case, the first operation instruction block sent to the second distributed database cluster system contains database operation instructions recorded in the master database between two synchronous clock signals and no consistency filtering is performed for these database operation instructions. The data synchronization module of the second distributed database cluster system normally synchronizes these database operation instructions to the corresponding master database of the second distributed database cluster system. However, the second distributed database cluster system performs global consistency filtering when cluster switching is performed for the service. This embodiment is suitable for scenarios where the disaster recovery system has high requirements on RPO indexes, since in this scheme data can be efficiently transmitted to a peer cluster and persisted, thereby ensuring that the RPO is as small as possible. A specific implementation of performing a global consistency filtering at the accepter end of data synchronization when cluster switching is performed for the service in this processing mode may refer to Embodiment 8.", + "p0074": "Real time global consistency scheme", + "p0075": "According to the real-time global consistency scheme, the first distributed database cluster system performs global consistency filtering on the first operation instruction block, encapsulates the database operation instructions, which satisfy the filtering condition, in the first operation instruction block into a second operation instruction block, and sends the second operation instruction block to a second distributed database cluster system. The filtering condition includes that the database operation instructions are in a global transaction submitted state. Through this scheme, the database operation instructions that have not been executed may be filtered from the first operation instruction block, and only the database operation instructions that have been executed are reserved and encapsulated into a second operation instruction block and sent to the second distributed database cluster system. In this way, the second distributed database cluster system synchronizes data that has been executed by the current first distributed database cluster system, so that the data synchronized to the backup cluster is always in a consistent state. This embodiment is suitable for scenarios where the disaster backup system has high requirements on RTO indexes. The service may take over the peer distributed database cluster at any time, and the RTO may be equal to zero. A specific implementation of performing global consistency filtering in real time at the synchronization initiator in this processing mode may refer to Embodiment 8.", + "p0076": "Considering that in the scenarios of incremental data subscription and incremental data extraction, the second distributed database cluster system has a filtering requirement for received data and only data objects satisfying a specific condition need to be sent to the second distributed database cluster system. Therefore, a filtering condition may be added, for example, the filtering condition may further include that the database operation instructions comply with a data object filtering rule corresponding to the second distributed database cluster system.", + "p0077": "For database operation instructions that have not been executed , the first distributed database cluster system buffers database operation instructions, which are in a global transaction unsubmitted state, in the first operation instruction block, adds the buffered database operation instructions to the next first operation instruction block corresponding to the current first master database, and then performs global consistency filtering on the next first operation instruction block.", + "p0078": "In order to perform global transaction management inside the first distributed database cluster system, so that the first distributed database cluster system can conveniently determine whether a database operation instruction is currently in a global submitted state or not, some improvement can be made to the first distributed database cluster system. The first distributed database cluster system manages a global transaction identification capable of uniquely identifying a global transaction, and records a state and a life cycle of each global transaction.", + "p0079": "The first distributed database cluster system manages and periodically requests a current active transaction list of each first master database, records the current active transaction list and a timestamp for acquiring the current active transaction list in a synchronization table corresponding to the first master database, and puts the timestamp and information of the current active transaction list in the clock synchronization instruction sent to the first master database.", + "p0080": "The first distributed database cluster system performs global consistency filtering on the first operation instruction block, and encapsulates database operation instructions, which satisfy a filtering condition, in the first operation instruction block into a second operation instruction block in the following manner: the first distributed database cluster system searches for a latest current active transaction list recorded in the synchronization table corresponding to a current first master database, removes database operation instructions corresponding to global transaction IDs in the latest current active transaction list, and encapsulates remaining database operation instructions into the second operation instruction block as the database operation instructions, which satisfy the filtering condition, in the global transaction submitted state.", + "p0081": "As an exemplary implementation, before data synchronization processing is performed, a synchronization position between the second distributed database cluster system at the peer end and the current first distributed database cluster system may be acquired. Before the first distributed database cluster system respectively encapsulates the database operation instructions recorded by each first master database between two adjacent clock synchronization instructions into a first operation instruction block, the method may further include: the first distributed database cluster system receives synchronization position request information sent by the second distributed database cluster system, wherein the synchronization position request information carries a synchronization position between the first distributed database cluster system and the second distributed database cluster system. The first distributed database cluster system may encapsulate the database operation instructions recorded by each first master database between two adjacent clock synchronization instructions into a first operation instruction block in the following manner: the first distributed database cluster system encapsulates the database operation instructions recorded by each first master database according to the synchronization position carried in the synchronization position request information.", + "p0082": "The embodiment further provides a storage medium, where a computer program is stored in the storage medium, and the computer program, when being executed, performs the method described in this embodiment.", + "p0083": "Embodiment 4", + "p0084": "This embodiment describes a processing flow of an accepter end of data synchronization. Fig. 7 is a flowchart of a data synchronization method according to Embodiment 4 of the present disclosure. The data synchronization method may be applied to a second distributed database cluster system including a plurality of second database groups 44, wherein each second database group 44 of the plurality of second database groups 44 includes a second master database. The method may include the following operations S702 and S704.", + "p0085": "In S702, the second distributed database cluster system receives an operation instruction block sent by a first distributed database cluster system. The operation instruction block includes a first operation instruction block or a second operation instruction block. Database operation instructions, which are recorded by each first master database of the first distributed database cluster system between two adjacent clock synchronization instructions, are encapsulated in the first operation instruction block. Database operation instructions, which satisfy a filtering condition, in the database operation instructions recorded by each first master database of the first distributed database cluster system between two adjacent clock synchronization instructions are encapsulated in the second operation instruction block. The filtering condition includes that the database operation instructions are in a global transaction submitted state.", + "p0086": "In S704, the second distributed database cluster system executes database operation instructions in the received operation instruction block.", + "p0087": "Through the above solution, the second distributed database cluster system, serving as the accepter end of the data synchronization, may execute database operation instructions according to the operation instruction block sent by the first distributed database cluster system of the initiator end of the data synchronization, so as to complete the data synchronization procedure. Among the operation instructions recorded in each master database of the first distributed database cluster system, the database operation instructions recorded between two adjacent clock synchronization instructions are carried the transmitted operation instruction block. Thus, it is ensured that for each master database, the data packets based on which the data synchronization is performed all have the same time reference point. Thus, the problem in the related art that data synchronized by different data nodes cannot achieve global time consistency can be solved. The data records for synchronization of different data nodes are ensured to be based on the same time reference point, thereby guaranteeing global consistency in terms of data records.", + "p0088": "As an exemplary embodiment, the second distributed database cluster system may execute the database operation instructions in the received operation instruction block in the following manner:", + "p0089": "the second distributed database cluster system persists the database operation instructions in the received operation instruction block into a playback file; and the second distributed database cluster system parses the playback file, splits the database operation instructions in the playback file by lines, and puts the database operation instructions into different working threads for processing.", + "p0090": "As an exemplary implementation, the second distributed database cluster system may parse the playback file, split the database operation instructions in the playback file by lines, and put the database operation instructions into different working threads for processing in the following manner:", + "p0091": "the second distributed database cluster system splits the database operation instructions in the received operation instruction block into different data lines according to table names and primary keys, wherein database operation instructions in a same data line compose a transaction according to a sequence; and the second distributed database cluster system puts database operation instructions of different transactions into different working threads for processing.", + "p0092": "In order to get prepared for the task restoration resulted from playback failure, as an exemplary implementation, after the second distributed database cluster system executes the database operation instructions in the received operation instruction block successfully, the method may further include: the second distributed database cluster system persists a position where the database operation instructions are executed successfully into a file.", + "p0093": "For the final global consistency scheme, the second distributed database cluster system may perform global consistency filtering cluster switching is performed for the service. Considering that the second distributed database cluster system may serve as a backup distributed database cluster system, in a case where the second distributed database cluster system needs to become a master distributed database cluster system in place of the first distributed database cluster system, the method may further include:", + "p0094": "the second distributed database cluster system acquires a synchronization table corresponding to each second master database, determines a timestamp of a latest synchronization for each second master database according to the synchronization table, and determines an earliest timestamp from the timestamp of the latest synchronization for each second master database as a switching reference time point; for each second master database for which the timestamp of the latest synchronization is later than the switching reference time point, the second distributed database cluster system rolls data of the second master database back to a recorded state at the switching reference time point, and restores the data of the second master database to a globally consistent state.", + "p0095": "As an exemplary implementation, the second distributed database cluster system rolls data of the second master database back to a recorded state at the switching reference time point, and restores the data of the second master database to a globally consistent state in the following manner:", + "p0096": "the second distributed database cluster system generates a reverse database operation instruction for each database operation instruction after the switching reference time point, and executes the reverse database operation instruction; and the second distributed database cluster system generates a reverse database operation instruction for each database operation instruction in a global transaction unsubmitted state at the switching reference time point, and executes the reverse database operation instruction.", + "p0097": "This embodiment further provides a storage medium, where a computer program is stored in the storage medium, and the computer program, when being executed, performs the method described in this embodiment.", + "p0098": "Embodiment 5", + "p0099": "This embodiment describes a distributed database data synchronization apparatus and method that ensure global consistency. Fig. 8 is a schematic diagram showing the architecture of a distributed database data synchronization apparatus for ensuring global consistency according to Embodiment 5 of the present disclosure. As shown in Fig. 8 , the distributed database data synchronization apparatus for ensuring global consistency in the embodiment of the present disclosure includes modules described as follows.", + "p0100": "A distributed database cluster is constituted by a plurality of database groups 80. Each database group 80 is constituted by a plurality of independent database nodes, data in the database nodes within a database group is completely consistent, and only one database node in the database group provides a writing service to the outside device, and other data nodes synchronize the data to the local through a replication technology between the databases. A database function is provided to an application in a data service manner, and a unified standard interface such as an SQL is provided to the application.", + "p0101": "A synchronization clock module 82 is configured to periodically send an SQL statement to each master data node in the distributed database, and the information carried in the SQL statement includes a timestamp of the synchronization clock module 82 and a current global active transaction list.", + "p0102": "A global transaction management module 84 is configured to uniformly manage a globally unique transaction ID, which uniquely identifies a global transaction, in a distributed environment, and meanwhile record a state and a life cycle of each global transaction. Each transaction in the cluster applies to the global transaction management module 84 to create a unique transaction ID, and is inserted into data lines.", + "p0103": "A management module 86 is configured to uniformly manage networking information of the distributed database cluster.", + "p0104": "A consistency filtering module 88 performs global consistency filtering on the data by using the global active transaction ID and filters out SQL statements containing a specified data object according to the filtering rule.", + "p0105": "A data synchronization module 810 is configured to collect a log of a master database node in the distributed database, generate SQL statements by parsing the log, package and transmit an SQL statement block message, persist the SQL message and perform parallel playback of the SQL lines.", + "p0106": "Based on the above architecture, in the data synchronization process, a synchronization table is firstly established in the active and backup clusters, and the synchronization table mainly includes two types of information: timestamp information and a global active transaction list. The synchronization clock module 82 periodically requests the current active transaction list from the global transaction management module 84. After acquiring the response from the global transaction management module 84, the synchronization clock module 82 sends an insertion statement to all the master databases in the master cluster to insert the timestamp for acquiring the current active global transaction list and the information of the current active global transaction list. The data synchronization module 810 of the master cluster periodically collects the log files of the master data nodes in the master cluster. After receiving a synchronization request from the backup cluster, the data synchronization module 810 inversely parses the log files to generate the SQL statements, segments the SQL statement stream according to the synchronization clock, performs consistency filtering and specified rule-based filtering on the SQL statements by using the consistency module, and finally encapsulates the SQL statements after the filtering in a sending queue. A sending thread then sends the messages in the sending queue to the data synchronization module 810 in the backup cluster. After receiving the SQL block message, the data synchronization module 810 in the backup cluster places the SQL messages into a receiving queue in the memory, invokes a writing thread to persist the SQL statement block in the receiving queue into the file, then starts a playback thread to split the statement block in the file into several tasks, and sends the data to a working thread to synchronize the data to the database node of the backup cluster.", + "p0107": "Through the above process, the log file of each data node in the master cluster is sliced according to a unified time within the cluster, and the transactions in the slices are processed uniformly on the main/backup cluster according to different service scenarios to make the transaction to simultaneously succeed or simultaneously fail in the main and backup clusters, thereby ensuring that the data on the backup cluster are in real-time consistency or final consistency. On this basis, the backup cluster may distribute the transactions in the slices according to the table name and the primary key value, and multi-thread execution can be performed in parallel, thereby solving the problem of low playback performance caused by insufficient concurrent shortage.", + "p0108": "Fig. 9 is a detailed working flowchart of a data synchronization method according to Embodiment 5 of the present disclosure. As shown in Fig. 9 , the distributed database data synchronization method for ensuring global consistency includes the following operations S902 to S918.", + "p0109": "In S902, when the data synchronization module 810 of the master cluster is started, a task thread for periodically acquiring metadata is initialized, and the database node connected to the data synchronization module 810 is ensured to be the master node in the database group 80. A boot thread reports the status to the management node periodically. When the data synchronization module 810 of the backup cluster is started, a concurrent working thread pool is initialized according to a configuration file, a message writing thread, an SQL playback thread and a garbage cleaning thread are initialized. After the initialization, a log synchronization position is firstly sent to the data synchronization module 810 of the master cluster.", + "p0110": "In S904, the data synchronization module 810 of the master cluster periodically sends a cluster networking data acquisition/synchronization request to the management module 86, and after receiving the response from the management module 86, compares the same with the data in the memory. If the data in the memory is empty, the memory structure is initialized with the current data. If there is data in the memory, whether the data is consistent is compared according to the sequence of the database group 80, and if the data is consistent, no processing is conducted, and if the data is not consistent, it is determined whether the data information is updated immediately according to whether the database node is connected.", + "p0111": "In S906, after the data synchronization module 810 of the master cluster starts and synchronizes the networking information of the clusters, the data synchronization module 810 of the master cluster waits for the data synchronization module 810 of the backup cluster to push the synchronization position request information, and after receiving the synchronization position information, starts a new task according to the current position.", + "p0112": "In S908, the data synchronization module 810 of the master cluster initiates a database connection according to the synchronization position, initializes the memory for parsing the log file, receives the log information pushed by the database, and then enters the SQL statement block segmentation process described in Embodiment 6. The SQL statement block is sent to the consistency filtering module 88 according to practical deployment.", + "p0113": "In S910, global consistency filtering and rule filtering are performed on the SQL statement block. For specific flow, reference may be made to Fig. 12 or Fig. 13 in Embodiment 8.", + "p0114": "In S912, when receiving the SQL statement block request sent by the data synchronization module 810 of the backup cluster, the data synchronization module 810 of the master cluster firstly determines whether the message needs to be retransmitted, if the message needs to be retransmitted, the data synchronization module 810 of the master cluster directly sends the messages in the sending queue, and if the message does not need to be retransmitted, messages are extracted from the sending buffer queue and then sent to the data synchronization module 810 of the master cluster.", + "p0115": "In S914, after receiving the message packet, the data synchronization module 810 of the backup cluster puts the message into the receiving list. Packet checking is performed. Then the SQL statement block message is put into a writing queue, the writing thread persists the message from the writing queue into the SQL playback file, and then checks whether there is a message blocked in the receive queue. If there is a message blocked in the receive queue, the suspended thread is woken up, otherwise, whether the current message group ends is judged, and the data synchronization position is updated and the writing queue is read cyclically.", + "p0116": "In S916, the SQL playback thread of the data synchronization module 810 of the backup cluster parses the SQL playback file, and parses the transaction statement blocks between the two synchronization clocks and splits the transaction statement blocks according to lines. For a specific method, reference may be made to the SQL parallel processing flow described in Embodiment 7. After the statement blocks are played back successfully in parallel, the data playback position is persisted in the file so that the data playback position can be used when the playback is abnormal.", + "p0117": "In S918, the data cleaning thread of the data synchronization module 810 of the backup cluster periodically cleans the data that has exceeded the saving time and has been successfully played back.", + "p0118": "In summary, the method provides a unified clock for the whole cluster through the synchronization clock module 82, and performs unified slicing processing on the data. The consistency filtering module 88 performs global consistency filtering on the data of the slices. Meanwhile, requirements on disaster recovery indexes, such as RPO and RTO , as well as requirements of services on data filtering and extraction, are considered, Different orchestrations of the synchronization clock module 82, the global transaction management module 84, the consistency filtering module 88, and the data synchronization module 810 can efficiently satisfy a plurality of service scenarios, such as multi-active in the same city, disaster recovery in different locations, data subscription and data extraction. Meanwhile, global consistency between data clusters in a distributed environment is ensured, so that the clusters in different locations can quickly take over services. The solution is a function with extremely useful characteristic, and can greatly facilitate the promotion of the distributed database in the financial field, the Internet field and the education field.", + "p0119": "Embodiment 6", + "p0120": "Fig. 10 is a schematic diagram of splitting and encapsulating of SQL statements according to Embodiment 6 of the present disclosure. As shown in Fig. 10 , the procedure is described as follows.", + "p0121": "The synchronization clock module 82 periodically sends a synchronization clock SQL statement to the database. The corresponding statement will eventually be recorded in the database log.", + "p0122": "After receiving the log pushed by the database, the data synchronization module 810 parses the log, and segments the application SQL by using the synchronization clock SQL statement. The specific implementation manner is that all SQLs between two synchronization clock SQL statements are considered as a message block, and are encapsulated into an SQL statement block.", + "p0123": "After the statement block is filtered, the statement block is packed and put into the sending buffer queue, and then the sending thread sends the SQL statement message from the buffer queue to the data synchronization module 810 at the peer end.", + "p0124": "Embodiment 7", + "p0125": "Fig. 11 is a flowchart of parallel processing of SQL statement lines according to Embodiment 7 of the present disclosure. As shown in Fig. 11 , the SQL statement block is divided into a plurality of transactions according to data lines, and the specific implementation is described as follows.", + "p0126": "The data lines are selected according to table names and primary keys, the SQLs of the same data line compose a transaction according to a sequence, and the SQLs of different data lines are sent to different working threads for processing.", + "p0127": "The working thread processes a batch of SQL statements in parallel to ensure successful playback, and then reads the next SQL statement block.", + "p0128": "Embodiment 8", + "p0129": "This embodiment describes the specific process of a real-time global consistency filtering procedure and a final global consistency filtering procedure.", + "p0130": "Fig. 12 is a flowchart of real-time global consistency filtering according to Embodiment 8 of the present disclosure, and the flowchart is described as follows.", + "p0131": "In S1202, a latest record is found from a synchronization table.", + "p0132": "In S1204, a global active transaction list at the latest synchronization is acquired from the record.", + "p0133": "In S1206, the SQL statement block is filtered by using the global active transaction ID.", + "p0134": "In S1208, whether there is a configuration data object filtering rule for the SQL statements submitted globally in the statement block is determined, if yes, operation S1210 is performed, if no, the SQL statements are submitted to the sending queue.", + "p0135": "In S1210, the involved SQL statements are filtered according to the filtering rule, and then the remaining SQL statements are submitted to a sending queue.", + "p0136": "In S1212, SQL statements that are not globally submitted are stored in a buffer queue.", + "p0137": "In S1214, before filtering the next batch of statements, the SQL statements in the buffer queue are combined into the statement block.", + "p0138": "Fig. 13 is a flowchart of final global consistency filtering according to Embodiment 8 of the present disclosure, and the flowchart is described as follows.", + "p0139": "In S1302, when it is necessary to perform master-backup cluster switching, the backup cluster finds the clock record of the latest synchronization from the synchronization table of all data nodes.", + "p0140": "In S1304, all the clock records of the latest synchronization are compared to find the time of the synchronization clock records with the earliest timestamp as the consistency point of switching.", + "p0141": "In S1306, all data nodes receiving a synchronization clock message later than the moment T0 are found.", + "p0142": "In S 1308, all reverse SQL statements for restoring back to the moment T0 are generated by parsing the log file, and the SQL statements are executed to roll the node data back to the moment of receiving the moment T0 synchronization clock message, so that the data of all the nodes in the cluster are aligned to the same time point .", + "p0143": "In S1310, the corresponding SQL statements in the log file of the backup cluster data node are filtered out by using the current global active transaction ID at the time point in the synchronization clock table. Although these statements are submitted at a single point, the transaction is not globally submitted. These transactions are in a global active state.", + "p0144": "In S1312, reverse SQL statements are generated and executed, and finally the backup cluster of the distributed database reaches a globally consistent state.", + "p0145": "Examples 9 to 12 below describe specific processing flows when the method of the present disclosure is applied in different scenarios.", + "p0146": "Embodiment 9", + "p0147": "This embodiment describes distributed database asynchronous data synchronization under a high performance mode .", + "p0148": "Fig. 14 is a system architecture diagram for data synchronization of a distributed database cluster under a high performance mode according to Embodiment 9 of the present disclosure.", + "p0149": "As shown in Fig. 14 , in this embodiment, there are two cluster systems. The master distributed database cluster will synchronize data to the backup cluster, and the networking of the two clusters is consistent. In the data synchronization process, the consistency filtering is not performed, and the global consistency filtering is performed only when the service switches the cluster. The processing of the distributed database asynchronous data synchronization under the high performance mode includes the following operations.", + "p0150": "1. The synchronization clock module 82 sends a synchronization SQL statement to the database. 2. The database pushes the log file to the data synchronization module 810 periodically. 3. After receiving the log pushed by the database, the data synchronization module 810 parses the log, and takes all SQLs between two synchronization SQLs as a message block and encapsulates these SQLs into an SQL statement block. After the SQL statement block is packed and compressed, the SQL statement block is put into the sending buffer queue, and then the sending thread sends the SQL statement message from the buffer queue to the data synchronization module 810 at the peer end. 4. After receiving the message packet, the data synchronization module 810 of the backup cluster puts the message into the receiving list. After checking is conducted on the message packet, the SQL statement block message is persisted to the SQL playback file, and then the SQL playback thread parses the SQL playback file, and the transaction statement blocks between two synchronization messages are parsed and split according to lines, and the SQL statements are played back in parallel, thereby achieving the data synchronization. 5. The consistency filtering module 88 is enabled only when master and backup cluster switching is required. The filtering operations are shown in Fig. 13 .", + "p0151": "This embodiment is suitable for scenarios where the disaster recovery system has high requirements on RPO indexes, since in this scheme data can be efficiently transmitted to a peer cluster and persisted, thereby ensuring that the RPO is as small as possible.", + "p0152": "Embodiment 10", + "p0153": "This embodiment describes distributed database cluster data synchronization under a high performance mode .", + "p0154": "Fig. 15 is a system architecture diagram for data synchronization of a distributed database cluster under a high performance mode according to Embodiment 10 of the present disclosure.", + "p0155": "As shown in Fig. 15 , in this embodiment, there are two database cluster systems. The master database cluster will synchronize data to the backup cluster, and the networking of the two clusters is consistent. In the data synchronization process, consistency filtering is performed, so that the data synchronized to the backup cluster is constantly in the consistency state. The processing in the distributed database cluster data synchronization under the high performance mode includes the following operations.", + "p0156": "1. The synchronization clock module 82 sends synchronization SQL statements to the database. 2. The database pushes the log file to the data synchronization module 810 periodically. 3. After receiving the log pushed by the database, the data synchronization module 810 parses the log, and takes all SQLs between two synchronization SQLs as a message block and encapsulates these SQLs into an SQL statement block. 4. The consistency filtering module 88 filters out statements in a global transaction submitted state in the statement block by using the global active transaction ID, and reference is made to Fig. 12 for a specific process. 5. After packaging and compressing the SQL statement block that has been in the global transaction submitted state, the SQL statement block is put into the sending buffer queue, and then the sending thread takes out the SQL statement message from the buffer queue and sends the SQL statement message to the data synchronization module 810 at the peer end. 6. After receiving the message packet, the data synchronization module 810 of the backup cluster puts the message into the receiving list. After checking is conducted on the message packet, the SQL statement block message is persisted to the SQL playback file, and then the SQL playback thread parses the SQL playback file, and the transaction statement blocks between two synchronization messages are parsed and split according to lines, and the SQL statements are played back in parallel, thereby achieving the data synchronization. Since each synchronized SQL statement block is globally consistent, the data in the backup cluster database is always in a globally consistent state.", + "p0157": "This embodiment is suitable for scenarios where the disaster backup system has high requirements on RTO indexes. The service may take over the distributed database cluster at the peer end at any time, and the RTO may be equal to zero.", + "p0158": "Embodiment 11", + "p0159": "This embodiment describes data synchronization under incremental data subscription based on a distributed database.", + "p0160": "Fig. 16 is a schematic diagram showing the system architecture of a distributed database quasi-real-time backup according to Embodiment 11 of the present disclosure. As shown in Fig. 16 , in this embodiment, there are two database clusters. A remote database cluster serves as a database cluster subscribing to the service system cluster, and the networking of the two clusters may be inconsistent. The process of quasi-real-time backup under incremental data subscription based on the distributed database includes the following operations.", + "p0161": "1. The synchronization clock module 82 periodically sends a synchronization SQL statement to the database. 2. The database pushes the log file to the data synchronization module 810 periodically. 3. After receiving the log pushed by the database, the data synchronization module 810 parses the log, and takes all SQLs between two synchronization SQLs as a message block and encapsulates these SQLs into an SQL statement block. 4. The consistency filtering module 88 buffers, according to the global active transaction ID, the SQL statements, which are globally active, in the statement block into the active statement queue, and merges these SQL statements into the statement block when filtering the next statement block. In this way, the filtered SQL statements are all statements in the global transaction submitted state. The involved data objects are then filtered according to subscription rules. 5. The filtered SQL statement blocks that are in the global transaction submitted state are packaged and compressed and put into the sending buffer queue, and then the sending thread extracts the SQL statement messages from the buffer queue and sends the SQL statement messages to the data synchronization module 810 at the peer end. 6. After receiving the message packet, the data synchronization module 810 of the remote cluster puts the message into the receiving list. After checking is conducted on the message packet, the SQL statement block messages are persisted into the SQL playback file. The SQL playback thread parses the SQL playback file, parses and splits by lines the transaction statement blocks between two synchronization messages, and plays back the SQL statements in parallel according to the networking condition of the cluster, thereby achieving the purpose of quasi-real-time data backup.", + "p0162": "Embodiment 12", + "p0163": "This embodiment describes data synchronization based on distributed database incremental data.", + "p0164": "Fig. 17 is a schematic diagram showing the system architecture of data extraction based on distributed database incremental data according to Embodiment 12 of the present disclosure. As shown in Fig. 17 , in this embodiment, there are two types of service systems: an Online Transactional Processing system that generates data, and an Online Analytical Processing system and a Data Warehouse system that are mainly used for data analysis services. A large data platform assembly used for structured data processing, and a distributed file system of Hadoop ) for online data analysis is illustrated in the figure. The OLTP system imports data into a large data platform for the large data platform to perform analysis and mining. Data extraction based on distributed database incremental data includes the following operations.", + "p0165": "1. The synchronization clock module 82 sends the synchronization SQL statement to the database periodically. 2. The OLTP type service database cluster periodically pushes the log file to the data synchronization module 810. 3. After receiving the log pushed by the database, the data synchronization module 810 parses the log, and takes all SQLs between two synchronization SQLs as a message block and encapsulates these SQLs into an SQL statement block. 4. The consistency filtering module 88 buffers, according to the global active transaction ID, the SQL statements, which are globally active, in the statement block into the active statement queue, and merges these SQL statements into the statement block when filtering the next statement block. In this way, the filtered SQL statements are all statements in the global transaction submitted state. The involved data objects are then filtered according to data extraction rules. 5. The filtered SQL statement blocks that are in the global transaction submitted state are packaged and compressed and put into the sending buffer queue, and then the sending thread extracts the SQL statement messages from the buffer queue and sends the SQL statement messages to the data synchronization module 810 at the large data platform end. 6. After receiving the message packet, the data synchronization module 810 at the large data platform puts the message into the receiving list. After checking is conducted on the message packet, the SQL statement block messages are persisted into the SQL playback file. The SQL playback thread parses the SQL playback file, parses and splits by lines the transaction statement blocks between two synchronization messages, and plays back the SQL statements into the data computation layer of the large data platform in parallel. The computation layer performs SQL computation, and updates the data computation result to the storage system of the large data platform.", + "p0166": "To sum up, according to the distributed database cluster system and the data synchronization method provided by the embodiments of the present disclosure, the application of the distributed database cluster in the fields of teaching, financial, government and enterprise, Internet, etc. can be enlarged, system resource investment is saved, system performance is improved, data consistency is ensured, and user experience is improved. The solution is beneficial for the promotion of distributed databases.", + "p0167": "It is apparent that those skilled in the art should know that each module or each step of the present invention may be implemented through a universal computing device. They may be concentrated in a single computing device or distributed in a network formed by multiple computing devices. In at least one exemplary embodiment, they may be implemented by program codes executable for the computing devices and thus may be stored in a storage device for execution with the computing devices. Moreover, in some cases, the shown or described operations may be executed in sequences different from those described here, or may form various integrated circuit modules respectively, or multiple modules or operations therein may form a single integrated circuit module for implementation. Therefore, the present invention is not limited to any specific hardware and software combination.", + "p0168": "The above is only the exemplary embodiment of the present invention and not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations.", + "features": { + "F1": "The distributed database cluster system comprises a first synchronization clock module, a first data synchronization module, and a plurality of first database groups, each comprising a first master database.", + "F2": "The first synchronization clock module is configured to periodically send a clock synchronization instruction to each first master database.", + "F3": "The first data synchronization module encapsulates database operation instructions, recorded by each first master database between two adjacent clock synchronization instructions, into a first operation instruction block.", + "F4": "The first master database writes received database operation instructions and clock synchronization instructions into its log file.", + "F5": "The first data synchronization module acquires the log file of each first master database, encapsulates instructions into the first operation instruction block, and sends it.", + "F6": "The distributed database cluster system includes a first consistency filtering module that receives the first operation instruction block and performs global consistency filtering.", + "F7": "The first consistency filtering module encapsulates filtered database operation instructions into a second operation instruction block and sends it to a peer system.", + "F8": "The filtering condition includes database operation instructions being in a global transaction submitted state or complying with a distributed database cluster system's filtering rule at the peer end.", + "F9": "The first consistency filtering module buffers database operation instructions in a global transaction unsubmitted state until a subsequent operation instruction block is received.", + "F10": "The first consistency filtering module adds buffered instructions to the operation instruction block and re-filters for global consistency when a next block is received.", + "F11": "The first data synchronization module is configured to send the first operation instruction block to a peer distributed database cluster system.", + "F12": "The system may include a global transaction management module managing global transaction identifiers and lifecycle states.", + "F13": "The first synchronization clock module periodically requests active transaction lists from each first master database and records timestamps.", + "F14": "The first consistency filtering module uses the latest active transaction list to filter and encapsulate database operation instructions into the second operation instruction block.", + "F15": "The first data synchronization module can receive synchronization position request information from a peer system and encapsulate instructions accordingly.", + "F16": "The first data synchronization module may execute database operation instructions in a received operation instruction block by persisting to a playback file and processing with working threads.", + "F17": "If the system acts as a backup system, it may use the first consistency filtering module to restore a master state.", + "F18": "In backup mode, the first consistency filtering module may roll data back to a consistent state using reverse instructions.", + "F19": "A second data synchronization module and a plurality of second database groups are part of another distributed database cluster system, with each group having a second master database.", + "F20": "The second data synchronization module receives operation instruction blocks from a peer system and executes them by persisting and processing with working threads.", + "F21": "The second consistency filtering module restores data of the second master database to a globally consistent state upon switch over.", + "F22": "A data synchronization method involves periodic clock synchronization, encapsulating operation instructions, and filtering, applied to the first distributed database cluster system.", + "F23": "Another data synchronization method involves receiving and executing operation instruction blocks, applied to the second distributed database cluster system." + } + } + }, + { + "FAN": "78090091", + "CPC": [ + "G06F-016/00", + "G06F-016/955", + "G06F-016/958", + "G06F-017/16", + "G06F-017/30", + "G06F-018/22", + "G06F-021/55/4", + "G06F-021/56/6", + "G06F-2221/2119", + "G06K-009/00/483", + "G06K-009/46/71", + "G06K-009/62/15", + "G06V-010/462", + "G06V-030/418", + "H04L-063/1416", + "H04L-063/1425", + "H04L-067/02" + ], + "Content": { + "title": "Anomaly detection for web document revision", + "pa01": "Aspects of the present disclosure include a system comprising a machine-readable storage medium storing at least one program and computer-implemented methods for detecting anomalies in revisions to a web document. Consistent with some embodiments, a method includes publishing, at a network-based content publication platform, a web document comprising a plurality of distinct elements generated using data received from a computing device of a user. The method further includes accessing an updated web document that was generated based on modifications to the published web document made by the user. The method further includes generating one or more anomaly scores based on a comparison of the updated web document to the published web document, and determining whether to allow publication of the updated web document based on a result of a comparison of the anomaly score to a threshold anomaly score.", + "c-en-0001": "A computer-implemented method comprising: accessing a published web document comprising content generated via a computing device associated with a user; receiving a request to revise the published web document to an updated web document, the request being transmitted from the computing device associated with the user, the updated web document being generated based on one or more modifications by the user to the content of the published web document; in response to the receiving of the request, generating one or more anomaly scores based on a comparison of the updated web document to the published web document, the one or more anomaly scores indicate how much the updated web document deviates from the content of the published web document; executing the request to revise the published web document by preventing publication of the updated web document based at least in part on the one or more anomaly scores indicating that the updated web document deviates from the content of the published web document beyond a threshold of change, the updated web document deviating from the content of the published web document beyond the threshold of change is indicative of the updated web document deviating too much from the content of the published web document to allow publication of the updated web document; and in response to the preventing publication, generating a message that indicates to the user that the one or more modifications are not allowed to be made to the published web document.", + "c-en-0002": "The method of claim 1 , wherein the generating of the one or more anomaly scores comprises generating an image similarity score based on a comparison of an updated image in the updated web document with a published image included in the published web page, wherein the published image is associated with a product listing that corresponds with a product for sale within an online marketplace.", + "c-en-0003": "The method of claim 2 , wherein the generating of the one or more anomaly scores further comprises: generating a text match score based on a comparison of updated text included in the updated web document with published text included in the published web document, wherein the published text is associated with the product listing that corresponds with the product for sale within the online marketplace; generating a deviation score based on a difference between an updated numerical value included in the updated web document and an average of numerical values associated with the published web document, and wherein the average of numerical values is associated with the product listing that corresponds with the product for sale within the online marketplace; and aggregating the text match score, the image similarity score, and the deviation score to generate the one or more anomaly scores.", + "c-en-0004": "The method of claim 1 , wherein generating the one or more anomaly scores comprises: performing a comparison of updated text included in the updated web document with published text included in the published web document; and generating a text match score based on the comparison, the text match score providing a measure of similarity between the updated text and the published text, wherein the one or more anomaly scores includes the text match score.", + "c-en-0005": "The method of claim 1 , further comprises: performing a comparison of a published image included in the published web document with a updated image included in the updated web document; and generating an image similarity score based on the comparison, the image similarity score providing a measure of similarity between the published image and the updated image, wherein the one or more anomaly scores includes the image similarity score.", + "c-en-0006": "The method of claim 5 , wherein the performing of the comparison comprises: extracting a first set of feature descriptors based on the published image, each feature descriptor of the first set of feature descriptors are indicative of a respective sub region within the published image; storing the first set of feature descriptors in a first matrix corresponding to the published image, the first matrix includes an array data structure comprising a collection of elements, each element is identified by at least one index or key; extracting a second set of feature descriptors based on the updated image; storing the second set of feature descriptors in a second matrix corresponding to the updated image; and comparing the first matrix to the second matrix, wherein the generating of the image similarity score is based on the comparing of the first matrix to the second matrix.", + "c-en-0007": "The method of claim 1 , wherein the generating of the one or more anomaly scores includes generating a numerical deviation score based on a difference between a first numerical value within the published web document and a second numerical value within the updated web document.", + "c-en-0008": "A system comprising: one or more processors; a non-transitory computer readable medium storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: accessing a published web document comprising a plurality of distinct elements generated via a computing device associated with a user; receiving a request to revise the published web document to an updated web document, the request being transmitted from the computing device associated with the user, the updated web document being generated based on one or more modifications by the user to at least one of the plurality of distinct elements of the published web document; in response to the receiving of the request, generating an anomaly score for each of the plurality of distinct elements based on a comparison of the updated web document to the published web document, each anomaly score for the plurality of distinct elements indicates how much a respective element of the updated web document deviates from a corresponding element of the plurality of distinct elements of the published web document; based at least in part on the respective element of the updated web document deviating from the corresponding element of the published web document being beyond a threshold of change, executing the request to revise the published web document by preventing publication of the respective element of the updated web document, the respective element of the updated web document deviating from the corresponding element of the published web document beyond the threshold of change is indicative of the respective element deviating too much from the corresponding element to allow publication of the respective element; and in response to the preventing publication, generating a message that indicates to the user that the one or more modifications are not allowed to be made to the published web document.", + "c-en-0009": "The system of claim 8 , wherein the plurality of distinct elements of the published web document include text, an image, and a numerical value within a product listing corresponding to a product for sale.", + "c-en-0010": "The system of claim 8 , wherein the generating the anomaly score comprises: performing a comparison of updated text included in the updated web document with published text included in the published web document, wherein the updated text is one element of the plurality of distinct elements; and generating a text match score based on the comparison, the text match score providing a measure of similarity between the updated text and the published text, wherein the anomaly score includes the text match score for the one element.", + "c-en-0011": "The system of claim 10 , wherein performing the comparison comprises using a cosine similarity algorithm to determine the similarity between the updated text and the published text that measures the cosine angle between two vectors represented by the updated text and the published text.", + "c-en-0012": "The system of claim 8 , wherein generating the anomaly score comprises: performing a comparison of a published image included in the published web document with a updated image included in the updated web document, the published image depicting a product for sale in an online marketplace; and generating an image similarity score based on the comparison, the image similarity score providing a measure of similarity between the published image and the updated image, wherein the anomaly score includes the image similarity score.", + "c-en-0013": "The system of claim 12 , wherein the performing of the comparison comprises: extracting a first set of feature descriptors based on the published image; storing the first set of feature descriptors in a first matrix corresponding to the published image; extracting a second set of feature descriptors based on the updated image; storing the second set of feature descriptors in a second matrix corresponding to the updated image; and comparing the first matrix to the second matrix, wherein the generating of the image similarity score is based on the comparing of the first matrix to the second matrix.", + "c-en-0014": "The system of claim 8 , wherein: the generating of the one or more anomaly scores includes generating a numerical deviation score based on a difference between a updated numerical value included in the updated web document and an average of numerical values associated with the web document, wherein; and the one or more anomaly scores includes the numerical deviation score.", + "c-en-0015": "The system of claim 8 , wherein the generating of the anomaly score comprises: generating, for a first element of the plurality of distinct elements, a text match score based on a comparison of updated text included in the updated web document with published text included in the published web document; generating, for a second element of the plurality of distinct elements, an image similarity score based on a comparison of an updated image included in the updated web document with a published image included in the published web page; generating, for a third element of the plurality of distinct elements, a numerical deviation score based on a difference between an updated numerical value included in the updated web document and an average of numerical values associated with the web document; and aggregating the text match score, the image similarity score, and the numerical deviation score to generate an aggregate anomaly score.", + "c-en-0016": "The system of claim 8 , further comprising: transmitting the updated web document for to a device of an administrator for manual review by the administrator; and refining one or more threshold anomaly scores based on a result of the manual review of the administrator.", + "c-en-0017": "The system of claim 16 , further comprising: receiving, from the device, an approval of the updated web document as the result of the manual review; and refining the one or more threshold anomaly score by increasing the one or more threshold anomaly scores based on the approval of the updated web document as the result of the manual review.", + "c-en-0018": "The system of claim 8 , further comprising: generating a message indicating that the updated web document is anomalous; and transmitting the message to a client device of the user responsible for providing the updated web document.", + "c-en-0019": "A non-transitory machine-readable storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising: publishing a web document comprising a plurality of distinct elements generated using data received from a computing device of a user; receiving a request to revise the published web document to an updated web document, the request being transmitted from the computing device associated with the user, the updated web document being based on one or more modifications by the user to the published web document made by the user using an interface presented on a computing device of the user, the updated web document including at least one user generated modification to a distinct element of the plurality of distinct elements of the published web document; in response to the receiving of the request, generating one or more anomaly scores based on a comparison of the updated web document to the published web document, the one or more anomaly scores indicate how much the updated web document deviates from the published web document; and executing the request to revise the published web document by preventing or allowing publication of the updated web document based at least in part on the updated web document deviating from the published web document beyond or not beyond an expected threshold of change, the updated web document deviating from the published web document beyond the expected threshold of change is indicative of the updated web document deviating too much from the published web document to allow publication of the updated web document, the updated web document not deviating from the published web document beyond the expected threshold of change is indicative of the updated web document not deviating too much from the published web document such that publication of the updated web document is allowed.", + "c-en-0020": "The non-transitory machine-readable storage medium of claim 19 , wherein the instructions further cause the machine to perform further operations comprising: based on the generating of the one or more anomaly scores, generating a decision tree for making classifications within a training model; transmitting the updated web document to a computer device associated with an administrative user for review; and refining the training model based on analyzing results, received from the computer device, of the review for inclusion in the decision tree, the refining of the decision tree includes increasing or decreasing the one or more anomaly scores.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a continuation of allowed U.S. patent application Ser. No. 15/188,532 entitled \u201cAnomaly Detection for Web Document Revision,\u201d filed Jun. 21, 2016, which is incorporated herein by reference in its entirety.", + "p0003": "TECHNICAL FIELD", + "p0004": "The present disclosure generally relates to machines configured to the technical field of special-purpose machines that facilitate management of digital content including computerized variants of such special-purpose machines and improvements to such variants, and to the technologies by which such special-purpose machines become improved compared to other special-purpose machines that facilitate management of digital content. In particular, the present disclosure addresses systems and methods to detect anomalous revisions to published digital content.", + "p0005": "Many online content publication platforms allow users to generate and publish content online in the form of a web document , which may be viewed by others users using a web browser or application. Each published web document is normally assigned a uniform resource identifier at or before the time of publication. Typically, these online content publication platforms allow users to revise content even once it has been published. Though some of the content of the web document may be updated, the URI typically remains unchanged. In some instances, allowing revisions to web documents while maintaining the same URI may be problematic for online content publication platforms.", + "p0006": "In an example, the content publication platform is an online marketplace that allows users to create web documents in the form of a product listing to offer their products for sale to other users. A user of the online marketplace may initially create a product listing for a first product that is highly demanded, and the online marketplace may then assign a URI to the product listing. The user may then completely revise the product listing to cover a second product with lower demand and desirability while the URI assigned to the product listing will remain the same, though the URI may still be associated with inventory and historical sales of the first product with a higher demand. In this way, the user can exploit the revision ability of the online marketplace to manipulate search ranks for undesirable products, conceal lower demand, manipulate historical sales numbers, or otherwise manipulate the consumer demand for a product. While this may prove to be beneficial for the individual user, this results in an overall degradation in quality of navigation, accuracy of information, and the overall performance and reputation of the online marketplace.", + "p0007": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0008": "Various ones of the appended drawings merely illustrate example embodiments of the present disclosure and cannot be considered as limiting its scope.", + "p0009": "FIG. 1 is a network diagram depicting a content publication platform having a client-server architecture configured for exchanging data over a network, according to example embodiments.", + "p0010": "FIG. 2 is a block diagram depicting various functional components of an anomaly detection system, which is provided as part of the content publication platform, according to example embodiments.", + "p0011": "FIG. 3 is an interaction diagram depicting example exchanges between a publication system and the content publication platform, according to example embodiments.", + "p0012": "FIG. 4A is an interface diagram illustrating a published web document, according to example embodiments.", + "p0013": "FIG. 4B is an interface diagram illustrating updated version of the published web document, according to example embodiments.", + "p0014": "FIGS. 5-8 is are flow charts illustrating example operations of the anomaly detection system in providing an anomaly detection service for the content publication platform, according to example embodiments.", + "p0015": "FIG. 9 is a flow chart illustrating example operations of the content publication system in providing a user generated content publication service, according to example embodiments.", + "p0016": "FIG. 10 is a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions for causing the machine to perform any one or more of the methodologies discussed herein may be executed.", + "p0017": "DETAILED DESCRIPTION", + "p0018": "Reference will now be made in detail to specific example embodiments for carrying out the inventive subject matter. Examples of these specific embodiments are illustrated in the accompanying drawings. It will be understood that these examples are not intended to limit the scope of the claims to the illustrated embodiments. On the contrary, they are intended to cover alternatives, modifications, and equivalents as may be included within the scope of the disclosure. In the following description, specific details are set forth in order to provide a thorough understanding of the subject matter. Embodiments may be practiced without some or all of these specific details.", + "p0019": "Aspects of the present disclosure involve a system and methods for detecting anomalies in revisions to a web document. As used herein, an \u201canomaly\u201d includes modifications to a web document that alter the web document beyond an expected threshold of change. In example embodiments, the web document is a web page such as an online marketplace listing for a product. The inventive subject matter of the present disclosure is applicable to any online content publication platform that allows users to revise published content. In one example, the online content publication platform is an online marketplace.", + "p0020": "Example embodiments involve web documents that include multiple distinct elements such as images, text, and numerical values. In the example web document of a listing, the distinct elements may include an image of a product, a textual description of a product, and a price. In these example embodiments, a method may include accessing an updated version of a published web document. The method further includes comparing the individual parts of the published web document with corresponding parts of the updated document to generate various anomaly scores. For example, a published image included in the published web document may be compared to an updated image included in the updated web document to generate an image similarity score; published text included in the published web document may be compared to an updated text included in the updated web document to generate an text match score; and an updated numerical value included in the updated web document may be compared to a published numerical value to generate a numerical deviation score.", + "p0021": "The method may further include determining whether to publish the updated web document based on a comparison of the anomaly scores to threshold anomaly scores. In some embodiments, the system may prevent the publication of the updated web document if any one of the anomaly scores is above a threshold score. In other embodiments, the system may prevent the publication of the updated web document if a combination of the anomaly scores is above a threshold score. Additional aspects of the present disclosure include using machine-learning techniques to revise the threshold scores based on manual inspection of revised listings in which anomalies were detected.", + "p0022": "With reference to FIG. 1 , an example embodiment of a high-level client-server-based architecture 100 is shown. While FIG. 1 depicts the client-server-based architecture 100 , the present inventive subject matter is, of course, not limited to such an architecture, and could equally well find application in an event-driven, distributed, or peer-to-peer architecture system, for example. Further, to avoid obscuring the inventive subject matter with unnecessary detail, various functional components that are not germane to conveying an understanding of the inventive subject matter have been omitted from FIG. 1 . Moreover, it shall be appreciated that although the various functional components illustrated in FIG. 1 are discussed in a singular sense, multiple instances of any one of the various functional components may be employed.", + "p0023": "A content publication platform 102 , in the example form of a network-based marketplace, provides server-side functionality via a network 104 ) to one or more client devices 110 . FIG. 1 illustrates, for example, a web client 112 , a client application 114 , and a programmatic client 116 executing on client device 110 . One or more portions of network 104 may be an ad hoc network, an intranet, an extranet, a virtual private network , a local area network , a wireless LAN , a wide area network , a wireless WAN , a metropolitan area network , a portion of the Internet, a portion of the public switched telephone network , a cellular telephone network, a wireless network, a WiFi network, a WiMax network, another type of network, or a combination of two or more such networks.", + "p0024": "The client device 110 may comprise, but is not limited to, a mobile phone, desktop computer, laptop, portable digital assistants , smart phones, tablets, ultra books, netbooks, laptops, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user may utilize to access the content publication platform 102 . In some embodiments, the client device 110 may comprise a display module to display information . In further embodiments, the client device 110 may comprise one or more of a touch screens, accelerometers, gyroscopes, cameras, microphones, global positioning system devices, and so forth. In one embodiment, the content publication platform 102 is a network-based marketplace that publishes publications comprising item listings of products available on the network-based marketplace.", + "p0025": "One or more users 106 may be a person, a machine, or other means of interacting with client device 110 . In example embodiments, the user 106 is not part of the client-server-based architecture 100 , but may interact with the client-server-based architecture 100 via the client device 110 or another means. For example, the user 106 may provide input to the client device 110 , and the input is communicated to the content publication platform 102 via the network 104 . In this instance, the content publication platform 102 , in response to receiving the input from the user 106 , communicates information to the client device 110 via the network 104 to be presented to the user 106 . In this way, the user 106 can interact with the content publication platform 102 using the client device 110 .", + "p0026": "The client device 110 may include one or more client applications 114 such as, but not limited to, a web browser, messaging application, electronic mail application, an e-commerce site application , and the like. In some embodiments, if the e-commerce site application is included in the client device 110 , then this application is configured to locally provide the user interface and at least some of the functionalities with the client application 114 configured to communicate with the content publication platform 102 , on an as needed basis, for data or processing capabilities not locally available . Conversely if the e-commerce site application is not included in the client device 110 , the client device 110 may use its web browser to access the e-commerce site hosted on the content publication platform 102 .", + "p0027": "An application program interface server 120 and a web server 122 are coupled to, and provide programmatic and web interfaces respectively to an application server 140 . The application server 140 may host a publication system 142 and an anomaly detection system 144 , each of which may comprise one or more modules or applications 114 and each of which may be embodied as hardware, software, firmware, or any combination thereof. The application server 140 is, in turn, shown to be coupled to a database server 124 that facilitate access to database 126 . In an example embodiment, the database 126 is a storage devices that stores information to be posted to the publication system 142 . The databases 126 may also store digital item information in accordance with example embodiments.", + "p0028": "Additionally, a third party application 132 , executing on a third party server 130 , is shown as having programmatic access to the content publication platform 102 via the programmatic interface provided by the API server 120 . For example, the third party application 132 , utilizing information retrieved from the content publication platform 102 , supports one or more features or functions on a website hosted by the third party.", + "p0029": "The publication system 142 provides a number of publication functions and services to users 106 that access the content publication platform 102 . For example, the publication system 142 provides interfaces that allow the user 106 to create and publish web documents using the client device 110 . The publication system 142 may further provide interfaces that allow the user 106 to modify various parts of the published web document.", + "p0030": "The anomaly detection system 144 is configured to monitor changes made by the user 106 to published web documents in order to detect anomalous updates. To this end, the anomaly detection system 144 compares updated web documents to previous or original versions of the web documents to generate various anomaly scores based on the degree to which the web document is modified. The anomaly detection system 144 may flag certain updated web documents based on these anomaly scores to prevent the publication system 142 from publishing the updated web document.", + "p0031": "While the publication system 142 and anomaly detection system 144 are shown in FIG. 1 to both form part of the content publication platform 102 , it will be appreciated that, in alternative embodiments, each system 142 and 144 may form part of a service or platform that is separate and distinct from the content publication platform 102 . In some embodiments, the anomaly detection system 144 may form part of the publication system 142 .", + "p0032": "FIG. 2 is a block diagram depicting various functional components of the anomaly detection system 144 , according to example embodiments. To avoid obscuring the inventive subject matter with unnecessary detail, various functional components that are not germane to conveying an understanding of the inventive subject matter have been omitted from FIG. 2 . However, a skilled artisan will readily recognize that various additional functional components may be supported by the anomaly detection system 144 to facilitate additional functionality that is not specifically described herein.", + "p0033": "As shown, the anomaly detection system 144 includes anomaly detectors 200 , a decision module 240 , and a refinement module 250 . The anomaly detectors 200 include a text anomaly detector 210 , an image anomaly detector 220 , and a numerical anomaly detector 230 . Each of the above referenced functional components of the anomaly detection system 144 are configured to communicate with each other ). Any one or more of functional components illustrated in FIG. 2 and described herein may be implemented using hardware or a combination of hardware and software. For example, any module described herein may configure a processor to perform the operations described herein for that module. Moreover, any two or more of these modules may be combined into a single module, and the functions described herein for a single module may be subdivided among multiple modules. Furthermore, according to various example embodiments, any of the functional components illustrated in FIG. 2 may be implemented together or separately within a single machine, database, or device or may be distributed across multiple machines, databases, or devices.", + "p0034": "The text anomaly detector 210 is responsible for detecting anomalies in revised web documents occurring as a result of modifications to text included in one or more prior published versions of the web documents. To this end, the text anomaly detector 210 is configured to compare modified text included in a modified web document to published text included in a published web document. Using the comparison as a basis, the text anomaly detector 210 generates a text match score that provides a measure of similarity between the modified text and the published text. In determining whether the modified text represents an anomaly, the text anomaly detector 210 compares the text match score to a threshold text match score. If the text match score transgresses the threshold text match score, the text anomaly detector 210 determines that an anomaly exists with respect to the modified text.", + "p0035": "The image anomaly detector 220 is responsible for detecting anomalies in revised web documents occurring as a result of modifications to images included in one or more prior published versions of the web documents. To this end, the image anomaly detector 220 is configured to compare a modified image included in a modified web document to a published image included in a published web document. Using the comparison as a basis, the image anomaly detector 220 generates an image similarity score that provides a measure of similarity between the modified image and the published image. In determining whether the modified image represents an anomaly, the image anomaly detector 220 compares the image similarity score to a threshold image similarity score. If the image similarity score transgresses the threshold image similarity score, the image similarity detector 220 determines that an anomaly exists with respect to the modified image.", + "p0036": "In some embodiments, the image anomaly detector 220 extracts keypoints from an original image, which are then stored in the database 126 . The updated image is compared with the original published image by individually comparing each feature from the new image to the stored features of the original image in the database 126 and finding candidate matching features based on Euclidean distance of their feature vectors. From the full set of matches, subsets of keypoints that agree on the object and its location, scale, and orientation in the new image are identified by the image anomaly detector 220 to filter out good matches.", + "p0037": "The numerical anomaly detector 230 is responsible for detecting anomalies in revised web documents occurring as a result of modifications to numerical values included in one or more prior published versions of the web documents. To this end, the numerical anomaly detector 230 is configured to compare a modified numerical value included in a modified web document to one or more numerical values included in one or more published web documents. In some embodiments, the numerical anomaly detector 230 compares the modified numerical value to a published numerical value included in the published version of the same web document. In other embodiments, the numerical anomaly detector 230 compares modified numerical value to an average of multiple numerical values included in associated published web documents. For example, the web document may include a price for a product, and the numerical anomaly detector 230 may compare a modified price for the product to an average price for the product calculated from multiple listings for the product.", + "p0038": "Using the comparison as a basis , the numerical anomaly detector 230 generates a numerical deviation score that indicates a deviation of the modified numerical value from the one or more published numerical values. In determining whether the modified numerical value represents an anomaly, the numerical anomaly detector 230 compares the numerical deviation score to a threshold numerical deviation score. If the numerical deviation score transgresses the threshold numerical deviation score, the numerical anomaly detector 230 determines that an anomaly exists with respect to the modified numerical value.", + "p0039": "Each of the text match score, image similarity score, and numerical deviation scores may be considered anomaly scores. The text anomaly detector 210 , the image anomaly detector 220 , and the numerical anomaly detector 230 provide the anomaly scores to the decision module 240 along with indications of whether an anomaly exists in respective portions of the modified web document. The decision module 240 uses the information provided by the text anomaly detector 210 , the image anomaly detector 220 , and the numerical anomaly detector 230 to determine whether to allow publication of the modified web document. In some embodiments, the decision module 240 may prevent publication of the modified web document if an anomaly exists in any one portion of the modified web document.", + "p0040": "In some embodiments, the decision module 240 aggregates respective anomaly scores to generate an aggregate anomaly score. For example, the decision module 240 may sum each respective anomaly score to generate the aggregate anomaly score. The decision module 240 may further compare the aggregate anomaly score to a threshold aggregate anomaly score to determine whether to allow publication of the updated web document. If the aggregate anomaly score transgresses the threshold anomaly score, the decision module 204 prevents publication of the modified web document.", + "p0041": "The refinement module 250 is configured to refine various threshold scores based on results of manual inspection of modified web documents determine to be anomalous. For example, upon deciding to prevent publication of a modified web document, the decision module 240 may flag the modified web document for manual inspection by an administrative user. If, in manually inspecting the modified web document, the administrative user determines that an anomaly does not exist, the refinement module 250 may revise one or more of the threshold anomaly scores such as by increasing the threshold.", + "p0042": "FIG. 3 is an interaction diagram depicting example exchanges between the publication system 142 and the anomaly detection system 144 , according to example embodiments. At operation 302 , the publication system 142 publishes a web document generated using data received from the client device 110 based on information supplied by the user 106 . Upon being generated, the web document is assigned a URI that uniquely identifies the web document. The web document may be accessed by and displayed on the client device 110 or other such device using the web client 112 , client application 114 , or the programmatic client 116 . For example, the client device 110 submits a request to the application server 140 for a particular document, and the application server 140 responds to the client device 110 with the web document and any other data needed for the client device 110 to display the web document.", + "p0043": "As example, FIG. 4A illustrates a published web document 400 in the exemplary form of a marketplace product listing. As shown, the web document 400 comprises multiple distinct elements including: text 402 \u2014a title of a product; image 404 \u2014an image of the product; and numerical value 406 \u2014a price of the product. The web document 400 further includes URI 408 that uniquely identifies the web document 400 .", + "p0044": "Returning to FIG. 3 , the publication system 142 provides, to the client device 110 , a user interface for revising the published web document in operation 304 . The user interface includes multiple fields that correspond to the multiple distinct elements of the published web document that allow the user 106 to modify each portion of the web document.", + "p0045": "At operation 306 , the publication system 142 generates an updated web document based on modifications to the published web document. As an example, FIG. 4B illustrates an updated web document 450 generated based on modifications to the web document 400 . As shown, the updated web document 450 includes the URI 408 , which is the same as in the web document 400 , although the text 402 has been changed to text 452 , the image 404 has been changed to image 454 , and the numerical value 406 has been changed to numerical value 456 .", + "p0046": "At operation 308 , the anomaly detection system 144 accesses the updated web document generated by the publication system 142 . At operation 310 , the anomaly detection system 144 analyzes the updated web document. In analyzing the updated web document, the anomaly detection system 144 compares the various updated portions of the updated web document to the various portions of the published web document to generates anomaly scores to use as basis in determining whether the updated web document is anomalous.", + "p0047": "At operation 312 , the anomaly detection system 144 determines whether the updated web document is anomalous. In other words, the anomaly detection system 144 determines whether the updated web document includes an anomaly as a result of one or more modifications made by the user 106 using the user interface provided at operation 304 . The determining of whether the updated web document is anomalous is based on the anomaly detection system 144 determining whether one or more anomaly scores transgresses a corresponding threshold anomaly score.", + "p0048": "If, at operation 312 , the anomaly detection system 144 determines that the updated web document is not anomalous, the anomaly detection system 144 allows publication of the updated web document at operation 314 , and the publication system 142 publishes the updated web document at operation 316 .", + "p0049": "If, at operation 312 , the anomaly detection system 144 determines that the updated web document is anomalous, the anomaly detection system 144 prevents publication of the updated web document at operation 318 . For example, the anomaly detection system 144 may instantiate a flag that, when read by the publication system 142 , causes the publication system 142 to end the publication process with respect to the updated web document.", + "p0050": "In response to the anomaly detection system 144 preventing publication of the updated web document, the publication system 142 generates a message at operation 320 to inform the user 106 that the modifications to the web document will not be allowed and the updated web document will not be published due to the anomaly being detected. At operation 322 , the publication system 142 transmits the message to the client device 110 of the user 106 . The publication system 142 may utilize any one of a number of message delivery networks and platforms to deliver the message to the client device 110 of the user 106 . For example, the publication system 142 may deliver push notifications , electronic mail , instant message , short message service , text, facsimile, or voice ) messages via the wired , plain old telephone service , or wireless networks.", + "p0051": "FIG. 5 is a flow chart illustrating example operations of the anomaly detection system 144 in performing a method 500 for providing an anomaly detection service for the content publication platform 102 , according to example embodiments. The method 500 may be embodied in computer-readable instructions for execution by one or more processors such that the operations of the method 500 may be performed in part or in whole by the anomaly detection system 144 ; accordingly, the method 500 is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method 500 may be deployed on various other hardware configurations and the method 500 is not intended to be limited to the anomaly detection system 144 .", + "p0052": "At operation 505 , the anomaly detection system 144 accessing a published web document . The published web document includes multiple distinct elements such as text, one or more images, and one or more numerical values. The distinct elements may be generated by or based on information from the user 106 of the content publication platform 102 . In an example, the published web document corresponds to a marketplace product listing and includes a textual description of the product, one or more images of the product, and a price for the product.", + "p0053": "At operation 510 , the anomaly detection system 144 accesses an updated web document . The updated web document is based on one or more modifications to the published web document. As with the published web document, the updated web document includes multiple distinct elements . The updated web document includes at least one user revision to a distinct element of the published web document, and in some instances may include at least one user revision to each part of the published web document.", + "p0054": "The updated web document may be or include a data object stored in the database 126 , and may be generated based on user input received from a user interface that allows the user 106 to edit published web documents. Further, both the published web document and the updated web document are assigned with the same URI 408 .", + "p0055": "At operation 515 , the anomaly detectors 200 generate one or more anomaly scores. Each of the one or more anomaly scores provide a measure of deviation of the updated web document from the published web document. The one or more anomaly scores may include one or more of a text match score, an image similarity score, and a numerical deviation score. Accordingly, the generating of the one or more anomaly scores may include: computing a text match score based on a comparison of updated text included in the updated web document with published text included in the published web document; computing an image similarity score based on a comparison of a updated image included in the updated web document with a published image included in the published web page; and computing a numerical deviation score based on a difference between an updated numerical value included in the updated web document and an average of numerical values associated with the web document.", + "p0056": "In some embodiments, the one or more anomaly scores include an aggregate anomaly score. Accordingly, in these embodiments, the generating of the one or more anomaly scores may additionally include aggregating the text match score, image similarity score, and numerical deviation score to generate the aggregate anomaly score.", + "p0057": "At operation 520 , the anomaly detection system 144 detects whether the updated web document includes an anomaly. The detecting of the anomaly in the updated web document includes comparing the one or more anomaly scores to one or more corresponding threshold anomaly scores. In some embodiments, the detecting of an anomaly includes any one of: determining the text match score transgresses a threshold text match score; determining the image similarity score transgresses a threshold image similarity score; or determining the numerical deviation score transgresses a threshold numerical deviation score. In embodiments in which the one or more anomaly scores include an aggregate anomaly score, the detecting of the anomaly in the updated web document may include determining the aggregate anomaly score transgresses a threshold anomaly score.", + "p0058": "If, at operation 520 , the anomaly detection system 144 detects an anomaly in the updated web document, the anomaly detection system 144 prevents publication of the updated web document at operation 525 . For example, the anomaly detection system 144 may instantiate a flag associated with the updated web document that stops the publication system 142 from publishing the updated web document.", + "p0059": "If, at operation 520 , the anomaly detection system 144 does not detect an anomaly in the updated web document, the anomaly detection system 144 allows publication of the updated web document in operation 530 . Following the example from above, the anomaly detection system 144 does not instantiate the flag associated with an anomaly, and in turn, the publication system 142 proceeds to publish the updated web document. In another example, the anomaly detection system 144 may instantiate a different flag that signals the publication system 142 to proceed with publishing the updated web document.", + "p0060": "As shown in FIG. 6 , the method 500 may include operations 605 , 610 , 615 , 620 , 625 , and 630 . In some example embodiments, operations 605 , 610 , 615 , 620 , 625 , and 630 included in the method 500 may be performed prior to or as part of operation 515 of method 500 , in which the anomaly detection system 144 generates one or more anomaly scores.", + "p0061": "At operation 605 , the text anomaly detector 210 compares published text included in the published web document with updated text included in the updated web document . At operation 610 , the text anomaly detector 210 generates a text match score based on the comparison of the published text with the modified text. The text match score provides a measure of similarity between the published text and the modified text.", + "p0062": "The text match score may, for example, be or include a cosine similarity score, which provides a measure of similarity between two vectors of an inner product space that measures the cosine angle between them. Accordingly, in some embodiments, the generating of the text match score may include applying a cosine similarity algorithm to two vectors\u2014a first vector corresponding to the published text and a second vector corresponding to the updated text.", + "p0063": "In a first example, the published text may include \u201cHeadset work Mobile phones Driving Sunglass Headphone BTGlass Wireless Bluetooth\u201d and the modified text may include \u201cHeadset work Mobile phones Driving Sunglass Headphone BTGlass Wireless Bluetooth\u2014with extra protection.\u201d In this example, the published text has been modified to include additional attributes, namely \u201cwith extra protection.\u201d Application of the cosine similarity algorithm in this example returns a cosine similarity score of 87.71.", + "p0064": "In a second example, the published text may include \u201cHeadset work Mobile phones Driving Sunglass Headphone BTGlass Wireless Bluetooth\u201d and the modified text may include \u201cHeadset Driving Sunglass Wireless Bluetooth Headphone BTGlass work Mobile phones.\u201d In this example, the published text has been modified by rearranging the order of some words. Application of the cosine similarity algorithm in this example returns a cosine similarity score of 99.99.", + "p0065": "In a third example, the published text may include \u201cHeadset work Mobile phones Driving Sunglass Headphone BTGlass Wireless Bluetooth\u201d and the modified text may include \u201cUniversal Qi Wireless Charger Receiver Charging Pad iphone android htc sony.\u201d In this example, the published text has been completely changed. Application of the cosine similarity algorithm in this example returns a cosine similarity score of 9.53.", + "p0066": "At operation 615 , the image anomaly detector 220 compares a published image included in the published web document with an updated image included in the updated web document. At operation 620 , the image anomaly detector 220 generates an image similarity score based on the comparison of the published image to the updated image. The image similarity score provides a measure of similarity between the published image and the updated image. Further details regarding operation 620 , according to some example embodiments, are discussed below in reference to FIG. 7 .", + "p0067": "At operation 625 , the numerical anomaly detector 230 generates a numerical deviation score based on an analysis of an updated numerical value included in the updated web document. In some embodiments, the numerical anomaly detector 230 generates the numerical deviation score by calculating the difference between the published numerical value and the updated numerical value.", + "p0068": "In other embodiments, the numerical anomaly detector 230 generates the numerical deviation score based on a comparison of the updated numerical value to an average of numerical values associated with the published web document. For example, the published web document may include a marketplace listing offering the product for sale, and the published numerical value may be a price for the product. In this example, the numerical anomaly detector 230 may determine an average price for the product based, for example, on other listings for the product published by the content publication platform 102 . The numerical anomaly detector 230 calculates the difference between an updated price for the product to the calculated average price for the product.", + "p0069": "At operation 630 , which is optional in some embodiments, the decision module 240 aggregates the text match score, the image similarity score, and the numerical deviation score to generate an aggregate anomaly score. For example, the decision module 240 may sum the text match score, the image similarity score, and the numerical deviation score to generate the aggregate anomaly score.", + "p0070": "As shown in FIG. 7 , the method 500 may include the additional operations 705 , 710 , 715 , 720 , and 725 . In some example embodiments, operations 705 , 710 , 715 , 720 , and 725 included in the method 500 may be performed prior to or as part of operation 620 of operation 515 of method 500 , in which the image anomaly detector 220 generates the image similarity score.", + "p0071": "At operation 705 , the image anomaly detector 220 extracts a first set of feature descriptors from the published image . The first set of feature descriptors comprise interesting key points from the published image that provide a \u201cfeature description\u201d of the published image. To perform reliable recognition, it is important that the features extracted from the original image be detectable even under changes in image scale, noise, and illumination. Such points usually lie on high-contrast regions of the image, such as object edges, for example. Similarly, features located in articulated or flexible objects may be unsuitable if any change in their internal geometry happens between two images in the set being processed. Accordingly, the image anomaly detector 220 may extract a large number of features from the images to reduce the contribution of the errors caused by these local variations in the average error of all feature matching errors.", + "p0072": "In some embodiments, the image anomaly detector 220 may make use of a scale-invariant feature transform algorithm to extract image feature descriptors. Using SIFT, the image anomaly detector 220 transforms the published image into a large collection of feature vectors, each of which is invariant to image translation, scaling, and rotation, partially invariant to illumination changes, and robust to local geometric distortion. Key locations for extraction may include maxima and minima of the result of difference of Gaussian functions applied in scale space to a series of smoothed and resampled images. The image anomaly detector 220 may discard low contrast candidate points and edge response points along an edge. The image anomaly detector 220 assigns dominant orientations to localized keypoints. The image anomaly detector 220 may then obtain image feature descriptors by considering pixels around a radius of the key location, blurring and resampling of local image orientation planes.", + "p0073": "At operation 710 , the image anomaly detector 220 stores the first set of feature descriptors in a first matrix that corresponds to the published image. A matrix is an array data structure comprising a collection of elements, each of which is identified by at least one index or key.", + "p0074": "At operation 715 , the image anomaly detector 220 extracts a second set of image feature descriptors from the updated image . Similar to the first set of image feature descriptors, the second set of image descriptors comprise interesting key points from the updated image that provide a \u201cfeature description\u201d of the updated image. The image anomaly detector 220 extracts the second set of image feature descriptors from the updated image in a similar manner to that discussed above with reference to the extraction of the first set of image descriptors from the published image. At operation 720 , the image anomaly detector 220 stores the second set of feature descriptors in a second matrix that corresponds to the updated image.", + "p0075": "At operation 725 , the image anomaly detector 220 compares the first matrix to the second matrix to ascertain the similarity between the published image and the updated image. If both images show the same object, then the first and second matrices will have similarity, otherwise the matrices will not be similar. Further, if A corresponds to the first matrix and B corresponds to the second matrix, A\u00b7B 1 should be equal to the identity matrix.", + "p0076": "Consistent with some embodiments, the operation of comparing the first matrix to the second matrix may include image feature descriptor matching between the first and second matrices. The best candidate match for each feature descriptor in the second matrix is found by identifying its nearest neighbor in the first matrix. The nearest neighbors are defined as the feature descriptors with minimum Euclidean distance from the given descriptor vector. The image anomaly detector 220 may, for example, utilize the Best-bin-first algorithm, which is a variation of the k-d tree algorithm, to identify the nearest neighbors.", + "p0077": "The image anomaly detector 220 generates the image match score based on the comparison of the first matrix to the second matrix. In an example, the image match score generated by the image anomaly detector 220 is equal to the number of image features descriptors matched between the two matrices divided by the total number of image feature descriptors.", + "p0078": "As shown in FIG. 8 , the method 500 may include the additional operations 805 , 810 , 815 , 820 , and 825 . In some example embodiments, operations 805 , 810 , 815 , 820 , and 825 included in the method 500 may be performed prior to or as part of operation 520 of method 500 , in which the anomaly detection system 144 determines whether an anomaly is detected in the updated web document.", + "p0079": "At operation 805 , the text anomaly detector 210 compares the text match score to a threshold text match score to determine whether the text match score transgresses the threshold text match score. The threshold text match score may be a default value set by an administrator of the content publication platform 102 or may be a value refined by the refinement module 250 using machine learning techniques. The threshold text match score amount may be a minimum or a maximum value. Thus, the text match score may be considered as transgressing the threshold text match score as a result of the text match score being higher than the maximum text match score or lower than a minimum text match score.", + "p0080": "If, at operation 805 , the text anomaly detector 210 determines that the text match score transgresses the threshold text match score, the text anomaly detector 210 determines that an anomaly exists in the updated web document with respect to the updated text, and the decision module 240 determines that an anomaly exists in the updated web document at operation 825 . If at operation 805 , the text anomaly detector 210 determines the text match score does not transgress the threshold text match score, the method continues to operation 810 .", + "p0081": "At operation 810 , the image anomaly detector 220 compares the image similarity score to a threshold image similarity score to determine whether the image similarity score transgresses the threshold image similarity score. The threshold image similarity score may be a default value set by an administrator of the content publication platform 102 or may be a value refined by the refinement module 250 using machine learning techniques. The threshold image similarity score amount may be a minimum or a maximum value. Thus, the image similarity score may be considered as transgressing the threshold image similarity score as a result of the image similarity score being higher than the maximum image similarity score or lower than a minimum image similarity score.", + "p0082": "If, at operation 810 , the image anomaly detector 220 determines that the image similarity score transgresses the threshold image similarity score, the image anomaly detector 220 determines that an anomaly exists in the updated web document with respect to the updated image, and the decision module 240 determines that an anomaly exists in the updated web document at operation 825 . If, at operation 810 , the image anomaly detector 220 determines the image similarity score does not transgress the threshold image similarity score, the method continues to operation 815 .", + "p0083": "At operation 815 , the numerical anomaly detector 230 compares the numerical deviation score to a threshold numerical deviation score to determine whether the numerical deviation score transgresses the threshold numerical deviation score. The threshold numerical deviation score may be a default value set by an administrator of the content publication platform 102 or may be a value refined by the refinement module 250 using machine learning techniques. The threshold numerical deviation score amount may be a minimum or a maximum value. Thus, the numerical deviation score may be considered as transgressing the threshold numerical deviation score as a result of the numerical deviation score being higher than the maximum numerical deviation score or lower than a minimum numerical deviation score.", + "p0084": "If, at operation 815 , the numerical anomaly detector 230 determines that the numerical deviation score transgresses the threshold numerical deviation score, the numerical anomaly detector 230 determines that an anomaly exists in the updated web document with respect to the updated numerical, and the decision module 240 determines that an anomaly exists in the updated web document at operation 825 . If, at operation 815 , the numerical anomaly detector 230 determines the numerical deviation score does not transgress the threshold numerical deviation score, the method continues to operation 820 where the decision module 240 determines that no anomaly has been detected in the updated web document.", + "p0085": "FIG. 9 is a flow chart illustrating example operations of the content publication platform 102 in performing a method 900 for providing a user generated content publication service, according to example embodiments. The method 900 may be embodied in computer-readable instructions for execution by one or more processors such that the operations of the method 900 may be performed in part or in whole by the content publication platform 102 ; accordingly, the method 900 is described below by way of example with reference thereto. However, it shall be appreciated that at least some of the operations of the method 900 may be deployed on various other hardware configurations and the method 900 is not intended to be limited to the content publication platform 102 .", + "p0086": "At operation 905 , the anomaly detection system 144 accesses a corpus of revised web documents . Each revised web document included in the corpus includes at least one revision to a portion of a web document published by the publication system 142 .", + "p0087": "At operation 910 , the anomaly detection system 144 generates anomaly scores for each revised web document. The anomaly detection system 144 may generate the anomaly scores consistent with the methodologies discussed above in reference to FIGS. 4-7 .", + "p0088": "At operation 915 , the refinement module 250 uses the generated anomaly scores to build a decision tree for classification of future revisions. The refinement module 250 also stores the decision tree as a training model, which is stored in the database 126 .", + "p0089": "At operation 920 , the anomaly detection system 144 receives a revision to a published web document. The revision may be based on input received from the user 106 via a user interface provided by the content publication platform 102 and displayed on the client device 110 .", + "p0090": "At operation 925 , the anomaly detection system 144 generates one or more anomaly scores for the revised web document. The anomaly detection system 144 generates the anomaly scores consistent with the methodologies discussed above in reference to FIGS. 4-7 . At operations 930 , the one or more anomaly scores are provided to the refinement module 250 for inclusion in the training model.", + "p0091": "At operation 935 , the anomaly detection system 144 detects whether an anomaly occurs in the revised web document based on the one or more anomaly scores. If, at operation 935 , the anomaly detection system 144 does not detect an anomaly in the revised web document, the anomaly detection system 144 allows the publication system 142 to publish the revised web document at operation 940 . At operation 940 , the anomaly detection system 144 also transmits the revised web document to a computer device of an administrative user for manual review. The manual review process allows the administrative user to manually assess whether the revised document is anomalous and provide an approval of the revised web document based on the manual assessment. In other words, the administrative user approves the revised web document if he determines the revised web document does not include an anomaly.", + "p0092": "At operation 945 , the content publication system platform 102 receives the results of the manual review . If the administrative user approves the revised web document, the content publication platform 102 informs the user 106 that the revision is allowed and that the revised web document is going to be published. For example, the content publication platform 102 may transmit a message to the client device 110 to inform the user 106 of such.", + "p0093": "If the administrative user fails to approve of the revised web document , the content publication platform 102 informs the user 106 that the revision is not allowed and that the revised web document will not be published, at operation 955 . For example, the content publication platform 102 may transmit a message to the client device 110 to inform the user 106 of such.", + "p0094": "The content publication platform 102 further provides the results of the manual review to the refinement module 250 for inclusion in the decision tree, and the refinement module 250 , in turn, refines the training model. Depending on the particular results of the manual review, the refinement of the training model may, for example, include refining one or more threshold anomaly scores such as the threshold text match score, the threshold image similarity score, or the numerical deviation score. The refining of the one or more threshold anomaly scores may include increasing or decreasing the threshold anomaly scores.", + "p0095": "If, at operation 935 , the anomaly detection system 144 detects an anomaly in the revised web document, the anomaly detection system 144 prevents the publication system 142 from publishing the revised web document at operation 960 . At operation 960 , the anomaly detection system 144 also transmits the revised web document to a computer device of an administrative user for manual review. The manual review process allows the administrative user to manually assess whether the revised document is anomalous and provide an approval of the revised web document based on the manual assessment.", + "p0096": "At operation 965 , the content publication platform 102 receives the results of the manual review . If the administrative user does not approve the revised web document , the content publication platform 102 informs the user 106 , at operation 955 , that the revision is not allowed and that the revised web document is not being published. For example, the content publication platform 102 may transmit a message to the client device 110 to inform the user 106 of such.", + "p0097": "If the administrative user approves of the revised web document 450 , the content publication platform 102 provides the results of the manual review to the refinement module 250 for inclusion in the decision tree, and the refinement module 250 , in turn, refines the training model. Depending on the particular results of the manual review, the refinement of the training model may, for example, include refining one or more threshold anomaly scores such as the threshold text match score, the threshold image similarity score, or the numerical deviation score. The refining of the one or more threshold anomaly scores may include increasing or decreasing the threshold anomaly scores.", + "p0098": "Machine Architecture", + "p0099": "FIG. 10 is a block diagram illustrating components of a machine 1000 , according to some example embodiments, able to read instructions from a machine-readable medium and perform any one or more of the methodologies discussed herein. Specifically, FIG. 10 shows a diagrammatic representation of the machine 1000 in the example form of a computer system, within which instructions 1016 for causing the machine 1000 to perform any one or more of the methodologies discussed herein may be executed. For example, the instructions 1016 may include executable code that causes the machine 1000 to execute any one of the methods 500 or 900 . These instructions transform the general, non-programmed machine into a particular machine programmed to carry out the described and illustrated functions of the publication system 142 and the anomaly detection system 144 in the manner described herein. The machine 1000 may operate as a standalone device or may be coupled to other machines. In a networked deployment, the machine 1000 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer network environment. By way of non-limiting example, the machine 1000 may comprise or correspond to a server computer, a client computer, a personal computer , a tablet computer, a laptop computer, a netbook, a set-top box , a personal digital assistant , an entertainment media system, a cellular telephone, a smart phone, a mobile device, a wearable device , a smart home device , other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing the instructions 1016 , sequentially or otherwise, that specify actions to be taken by the machine 1000 . Further, while only a single machine 1000 is illustrated, the term \u201cmachine\u201d shall also be taken to include a collection of machines 1000 that individually or jointly execute the instructions 1016 to perform any one or more of the methodologies discussed herein.", + "p0100": "The machine 1000 may include processors 1010 , memory/storage 1030 , and I/O components 1050 , which may be configured to communicate with each other such as via a bus 1002 . In an example embodiment, the processors 1010 , a reduced instruction set computing processor, a complex instruction set computing processor, a graphics processing unit , a digital signal processor , an application specific integrated circuit , a radio-frequency integrated circuit , another processor, or any suitable combination thereof) may include, for example, a processor 1012 and a processor 1014 that may execute the instructions 1016 . The term \u201cprocessor\u201d is intended to include a multi-core processor 1010 that may comprise two or more independent processors that may execute instructions contemporaneously. Although FIG. 10 shows multiple processors, the machine 1000 may include a single processor with a single core, a single processor with multiple cores , multiple processors with a single core, multiple processors with multiples cores, or any combination thereof.", + "p0101": "The memory/storage 1030 may include a memory 1032 , such as a main memory, or other memory storage, and a storage unit 1036 , both accessible to the processors 1010 such as via the bus 1002 . The storage unit 1036 and memory 1032 store the instructions 1016 embodying any one or more of the methodologies or functions described herein. The instructions 1016 may also reside, completely or partially, within the memory 1032 , within the storage unit 1036 , within at least one of the processors 1010 , or any suitable combination thereof, during execution thereof by the machine 1000 . Accordingly, the memory 1032 , the storage unit 1036 , and the memory of the processors 1010 are examples of machine-readable media.", + "p0102": "As used herein, \u201cmachine-readable medium\u201d means a device able to store instructions and data temporarily or permanently, and may include, but is not limited to, random-access memory , read-only memory , buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage ) and/or any suitable combination thereof. The term \u201cmachine-readable medium\u201d should be taken to include a single medium or multiple media able to store the instructions 1016 . The term \u201cmachine-readable medium\u201d shall also be taken to include any medium, or combination of multiple media, that is capable of storing instructions for execution by a machine , such that the instructions, when executed by one or more processors of the machine , cause the machine to perform any one or more of the methodologies described herein. Accordingly, a \u201cmachine-readable medium\u201d refers to a single storage apparatus or device, as well as \u201ccloud-based\u201d storage systems or storage networks that include multiple storage apparatus or devices. The term \u201cmachine-readable medium\u201d excludes signals per se.", + "p0103": "The I/O components 1050 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components 1050 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones will likely include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 1050 may include many other components that are not shown in FIG. 10 . The I/O components 1050 are grouped according to functionality merely for simplifying the following discussion, and the grouping is in no way limiting. In various example embodiments, the I/O components 1050 may include output components 1052 and input components 1054 . The output components 1052 may include visual components , a light emitting diode display, a liquid crystal display , a projector, or a cathode ray tube ), acoustic components , haptic components , other signal generators, and so forth. The input components 1054 may include alphanumeric input components , point based input components , tactile input components , audio input components , and the like.", + "p0104": "In further example embodiments, the I/O components 1050 may include biometric components 1056 , motion components 1058 , environmental components 1060 , or position components 1062 , among a wide array of other components. For example, the biometric components 1056 may include components to detect expressions , measure biosignals , identify a person , and the like. The motion components 1058 may include acceleration sensor components , gravitation sensor components, rotation sensor components , and so forth. The environmental components 1060 may include, for example, illumination sensor components , temperature sensor components , acoustic sensor components , or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment. The position components 1062 may include location sensor components receiver component), altitude sensor components , orientation sensor components , and the like.", + "p0105": "Communication may be implemented using a wide variety of technologies. The I/O components 1050 may include communication components 1064 operable to couple the machine 1000 to a network 1080 or devices 1070 via a coupling 1082 and a coupling 1072 respectively. For example, the communication components 1064 may include a network interface component or other suitable device to interface with the network 1080 . In further examples, the communication components 1064 may include wired communication components, wireless communication components, cellular communication components, near field communication components, Bluetooth\u00ae components , Wi-Fi\u00ae components, and other communication components to provide communication via other modalities. The devices 1070 may be another machine or any of a wide variety of peripheral devices ).", + "p0106": "Moreover, the communication components 1064 may detect identifiers or include components operable to detect identifiers. For example, the communication components 1064 may include radio frequency identification tag reader components, NFC smart tag detection components, optical reader components bar code, multi-dimensional bar codes such as Quick Response code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components . In addition, a variety of information may be derived via the communication components 1064 , such as location via Internet Protocol geolocation, location via Wi-Fi\u00ae signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.", + "p0107": "Transmission Medium", + "p0108": "In various example embodiments, one or more portions of the network 1080 may be an ad hoc network, an intranet, an extranet, a virtual private network , a local area network , a wireless LAN , a wide area network , a wireless WAN , a metropolitan area network , the Internet, a portion of the Internet, a portion of the public switched telephone network , a plain old telephone service network, a cellular telephone network, a wireless network, a Wi-Fi\u00ae network, another type of network, or a combination of two or more such networks. For example, the network 1080 or a portion of the network 1080 may include a wireless or cellular network and the coupling 1082 may be a Code Division Multiple Access connection, a Global System for Mobile communications connection, or another type of cellular or wireless coupling. In this example, the coupling 1082 may implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology , Evolution-Data Optimized technology, General Packet Radio Service technology, Enhanced Data rates for GSM Evolution technology, third Generation Partnership Project including 3G, fourth generation wireless networks, Universal Mobile Telecommunications System , High Speed Packet Access , Worldwide Interoperability for Microwave Access , Long Term Evolution standard, others defined by various standard-setting organizations, other long range protocols, or other data transfer technology.", + "p0109": "The instructions 1016 may be transmitted or received over the network 1080 using a transmission medium via a network interface device and utilizing any one of a number of well-known transfer protocols ). Similarly, the instructions 1016 may be transmitted or received using a transmission medium via the coupling 1072 to the devices 1070 . The term \u201ctransmission medium\u201d shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions 1016 for execution by the machine 1000 , and includes digital or analog communications signals or other intangible media to facilitate communication of such software.", + "p0110": "Modules, Components and Logic", + "p0111": "Certain embodiments are described herein as including logic or a number of components, modules, or mechanisms. Modules may constitute either software modules or hardware modules. A hardware module is a tangible unit capable of performing certain operations and may be configured or arranged in a certain manner. In example embodiments, one or more computer systems or one or more hardware modules of a computer system may be configured by software as a hardware module that operates to perform certain operations as described herein.", + "p0112": "In various embodiments, a hardware module may be implemented mechanically or electronically. For example, a hardware module may comprise dedicated circuitry or logic that is permanently configured or an application-specific integrated circuit ) to perform certain operations. A hardware module may also comprise programmable logic or circuitry that is temporarily configured by software to perform certain operations. It will be appreciated that the decision to implement a hardware module mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry may be driven by cost and time considerations.", + "p0113": "Accordingly, the term \u201chardware module\u201d should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured , or temporarily configured to operate in a certain manner and/or to perform certain operations described herein. Considering embodiments in which hardware modules are temporarily configured , each of the hardware modules need not be configured or instantiated at any one instance in time. For example, where the hardware modules comprise a general-purpose processor configured using software, the general-purpose processor may be configured as respective different hardware modules at different times. Software may accordingly configure a processor, for example, to constitute a particular hardware module at one instance of time and to constitute a different hardware module at a different instance of time.", + "p0114": "Hardware modules can provide information to, and receive information from, other hardware modules. Accordingly, the described hardware modules may be regarded as being communicatively coupled. Where multiple of such hardware modules exist contemporaneously, communications may be achieved through signal transmission . In embodiments in which multiple hardware modules are configured or instantiated at different times, communications between such hardware modules may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware modules have access. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware module may then, at a later time, access the memory device to retrieve and process the stored output. Hardware modules may also initiate communications with input or output devices, and can operate on a resource .", + "p0115": "The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented modules that operate to perform one or more operations or functions. The modules referred to herein may, in some example embodiments, comprise processor-implemented modules.", + "p0116": "Similarly, the methods described herein may be at least partially processor-implemented. For example, at least some of the operations of a method may be performed by one or more processors or processor-implemented modules. The performance of certain of the operations may be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In some example embodiments, the processor or processors may be located in a single location , while in other embodiments the processors may be distributed across a number of locations.", + "p0117": "The one or more processors may also operate to support performance of the relevant operations in a \u201ccloud computing\u201d environment or as a \u201csoftware as a service\u201d . For example, at least some of the operations may be performed by a group of computers , with these operations being accessible via a network and via one or more appropriate interfaces .", + "p0118": "Electronic Apparatus and System", + "p0119": "Example embodiments may be implemented in digital electronic circuitry, or in computer hardware, firmware, or software, or in combinations of them. Example embodiments may be implemented using a computer program product, for example, a computer program tangibly embodied in an information carrier, for example, in a machine-readable medium for execution by, or to control the operation of, data processing apparatus, for example, a programmable processor, a computer, or multiple computers.", + "p0120": "A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a standalone program or as a module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site, or distributed across multiple sites and interconnected by a communication network 104 .", + "p0121": "In example embodiments, operations may be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Method operations can also be performed by, and apparatus of example embodiments may be implemented as, special purpose logic circuitry .", + "p0122": "The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network 104 . The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments deploying a programmable computing system, it will be appreciated that both hardware and software architectures merit consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware , in temporarily configured hardware , or in a combination of permanently and temporarily configured hardware may be a design choice.", + "p0123": "Language", + "p0124": "Although the embodiments of the present disclosure have been described with reference to specific example embodiments, it will be evident that various modifications and changes may be made to these embodiments without departing from the broader scope of the inventive subject matter. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. The accompanying drawings that form a part hereof show by way of illustration, and not of limitation, specific embodiments in which the subject matter may be practiced. The embodiments illustrated are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be used and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. This Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of various embodiments is defined only by the appended claims, along with the full range of equivalents to which such claims are entitled.", + "p0125": "Such embodiments of the inventive subject matter may be referred to herein, individually and/or collectively, by the term \u201cinvention\u201d merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed. Thus, although specific embodiments have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific embodiments shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments. Combinations of the above embodiments, and other embodiments not specifically described herein, will be apparent, to those of skill in the art, upon reviewing the above description.", + "p0126": "All publications, patents, and patent documents referred to in this document are incorporated by reference herein in their entirety, as though individually incorporated by reference. In the event of inconsistent usages between this document and those documents so incorporated by reference, the usage in the incorporated references should be considered supplementary to that of this document; for irreconcilable inconsistencies, the usage in this document controls.", + "p0127": "In this document, the terms \u201ca\u201d or \u201can\u201d are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of \u201cat least one\u201d or \u201cone or more.\u201d In this document, the term \u201cor\u201d is used to refer to a nonexclusive or, such that \u201cA or B\u201d includes \u201cA but not B,\u201d \u201cB but not A,\u201d and \u201cA and B,\u201d unless otherwise indicated. In the appended claims, the terms \u201cincluding\u201d and \u201cin which\u201d are used as the plain-English equivalents of the respective terms \u201ccomprising\u201d and \u201cwherein.\u201d Also, in the following claims, the terms \u201cincluding\u201d and \u201ccomprising\u201d are open-ended; that is, a system, device, article, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim.", + "features": { + "F1": "A computer-implemented method includes accessing a published web document comprising content generated via a computing device associated with a user.", + "F2": "The method includes receiving a request to revise the published web document to an updated web document, the request being transmitted from the computing device associated with the user.", + "F3": "The updated web document is generated based on one or more modifications by the user to the content of the published web document.", + "F4": "In response to receiving the request, one or more anomaly scores are generated based on a comparison of the updated web document to the published web document.", + "F5": "The anomaly scores indicate how much the updated web document deviates from the content of the published web document.", + "F6": "The method prevents publication of the updated web document based at least in part on the anomaly scores indicating that the updated web document deviates from the content of the published web document beyond a threshold of change.", + "F7": "A message is generated indicating to the user that the modifications are not allowed to be made to the published web document in response to the prevention of publication.", + "F8": "Generating the anomaly scores includes generating an image similarity score based on a comparison of an updated image in the updated web document with a published image included in the published web document.", + "F9": "The published image is associated with a product listing that corresponds with a product for sale within an online marketplace.", + "F10": "Generating the anomaly scores further includes generating a text match score based on a comparison of updated text included in the updated web document with published text included in the published web document.", + "F11": "A deviation score is generated based on a difference between an updated numerical value and an average of numerical values associated with the published web document.", + "F12": "The text match score, image similarity score, and deviation score are aggregated to generate the anomaly scores.", + "F13": "Performing a comparison of updated text with published text results in generating a text match score, which provides a measure of similarity between the updated text and the published text.", + "F14": "Performing a comparison of a published image with an updated image results in generating an image similarity score, which provides a measure of similarity between the published image and the updated image.", + "F15": "Extracting feature descriptors from a published image, storing them in a matrix, and comparison with an updated image matrix generates the image similarity score.", + "F16": "The anomaly scores include a numerical deviation score based on the difference between a first numerical value in the published document and a second numerical value in the updated document.", + "F17": "A system includes one or more processors and a non-transitory computer-readable medium storing instructions causing the system to perform operations like accessing a published web document.", + "F18": "The system receives a request to revise the published web document, with the updated document generated by user modifications to distinct elements of the document.", + "F19": "An anomaly score for each distinct element is generated based on deviation from corresponding elements of the published document.", + "F20": "The system prevents publication of an element if it deviates beyond a threshold of change from its published counterpart and informs the user of non-allowable modifications.", + "F21": "Distinct elements in the published document include text, an image, and numerical values within a product listing.", + "F22": "Using cosine similarity to determine text similarity between vectors representing updated and published text.", + "F23": "Generating a numerical deviation score includes comparing an updated numerical value with an average of numerical values in the published document.", + "F24": "A message generation indicates deviations or anomalies to the user or administrator.", + "F25": "A decision tree is generated for classification within a training model based on anomaly scores, with administrator reviews refining the model." + } + } + }, + { + "FAN": "80155730", + "CPC": [ + "G06F-016/00", + "G06F-016/24575", + "G06F-016/24578", + "G06F-016/2465", + "G06F-016/9535", + "G06F-017/30/528", + "G06F-017/30/53", + "G06F-017/30/539", + "G06F-017/30/867", + "G06Q-030/02/82", + "G06Q-030/06/31", + "G06Q-050/01" + ], + "Content": { + "title": "Method and device for social platform-based data mining", + "pa01": "Provided is a method and device for social platform-based data mining. The method includes: acquiring one or more interest label dictionaries of one or more registered users on an information client and one or more first objects having followed relationship with the one or more registered users on the information client in a social platform; determining one or more first followed sets corresponding to the one or more registered users; constructing an interest model; acquiring one or more second objects having followed relationship with one or more newly registered users on the information and reading relationship information between the one or more newly registered users and the one or more second objects; determining a second followed set corresponding to the one or more newly registered users; and matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users.", + "c-en-0001": "A method for social platform-based data mining, comprising: acquiring, by a computer terminal, one or more interest label dictionaries of one or more registered users on an information client; acquiring, by the computer terminal, one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and reading relationship information between the one or more registered users and the one or more first objects; according to the one or more first objects having the followed relationship with the one or more registered users, determining, by the computer terminal, one or more first followed sets corresponding to the one or more registered users; according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, constructing, by the computer terminal, an interest model, wherein the interest model is used to characterize a corresponding relationship between one or more registered users having the same first followed set and an interest label; acquiring, by the computer terminal, one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and reading, by the computer terminal, relationship information between the one or more newly registered users and the one or more second objects; according to the one or more second objects having the followed relationship with the one or more newly registered user, determining, by the computer terminal, a second followed set corresponding to the one or more newly registered users; matching, by the computer terminal, the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model; and pushing recommended information to the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0002": "The method as claimed in claim 1 , wherein before acquiring the one or more interest label dictionaries of the one or more registered users on the information client, the method comprises: acquiring recommended information; extracting one or more interest labels of the recommended information from content of the recommended information; acquiring historical behavior data of the one or more registered users, wherein the historical behavior data is used to record operational behavior of the one or more registered users for the recommended information; determining one or more label weight values of the one or more interest labels according to the historical behavior data; and determining the one or more interest label dictionaries corresponding to the one or more registered users according to the one or more label weight values.", + "c-en-0003": "The method as claimed in claim 2 , wherein constructing the interest model according to the one or more interest label dictionaries of the one or more registered users and the first followed set comprises: performing filtering in the first followed set to acquire a third followed set corresponding to the one or more registered users, wherein a method for performing filtering in the first followed set comprises at least one of a data filtering method, an index filtering method, a condition filtering method, and an information filtering method; matching the one or more registered users based on the third followed set to generate a registered user set, wherein the registered user set comprises one or more registered users having the same third followed set; and generating a user set label dictionary corresponding to the registered user set according to the one or more interest label dictionaries of the one or more registered users comprised in the registered user set.", + "c-en-0004": "The method as claimed in claim 3 , wherein generating the user set label dictionary corresponding to the registered user set according to the one or more interest label dictionaries of the one or more registered users comprised in the registered user set comprises: acquiring a first user amount of the one or more registered users on the information client and a second user amount of the registered user set; calculating a weight distribution average value of each of the interest labels according to the one or more label weight values and a first user amount; calculating a set weight average value of each of the interest labels in a user set interest label dictionary according to the one or more label weight values and a second user amount of the one or more registered users in the registered user set; calculating a registered user set weight value of the one or more interest labels in the user set interest label dictionary according to the weight distribution average value and the set weight average value; successively comparing the registered user set weight value of the one or more interest labels in the user set interest label dictionary with a preset noise threshold; retaining the interest label corresponding to the registered user set weight value in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is greater than the preset noise threshold; and deleting the interest label corresponding to the registered user set weight value from the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is less than or equal to the preset noise threshold.", + "c-en-0005": "The method as claimed in claim 4 , wherein matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model comprises: performing filtering in the second followed set to acquire a fourth followed set corresponding to the one or more newly registered users, wherein the filtering method comprises at least one of a data filtering method, an index filtering method, a condition filtering method, and an information filtering method; matching the fourth followed set with the third followed set to determine the registered user set corresponding to the one or more newly registered users; and determining the one or more recommended interest labels of the one or more newly registered users according to the user set label dictionary of the registered user set corresponding to the one or more newly registered users.", + "c-en-0006": "The method as claimed in claim 5 , wherein after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the method further comprises: pushing recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0007": "The method as claimed in claim 4 , wherein the registered user set weight value V\u2032 of the one or more interest labels is determined by the following formula: V \u2032 = V / V base ; where V\u2032 denotes the registered user set weight value of an interest label i, V denotes the set weight average value of the interest label i, V base denotes the weight distribution average value of the interest label i.", + "c-en-0008": "The method as claimed in claim 4 , wherein after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the method further comprises: pushing recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0009": "The method as claimed in claim 3 , wherein after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the method further comprises: pushing recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0010": "The method as claimed in claim 2 , wherein after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the method further comprises: pushing recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0011": "The method as claimed in claim 1 , wherein after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the method further comprises: pushing recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0012": "A computer terminal for executing the program code of the steps provided by the method for social platform-based data mining as claimed in claim 1 .", + "c-en-0013": "A storage medium for storing the program code executed by the method for social platform-based data mining as claimed in claim 1 .", + "c-en-0014": "A device for social platform-based data mining, comprising a hardware processor configured to execute program units stored on a memory, the program units comprising: a first acquiring component arranged to acquire one or more interest label dictionaries of one or more registered users on an information client; a second acquiring component arranged to acquire one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and read the relationship information between the one or more registered users and the one or more first objects; a first determining component arranged to, according to the one or more first objects having the followed relationship with the one or more registered users, determine one or more first followed sets corresponding to the one or more registered users; a first processing component arranged to, according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, construct an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; a third acquiring component arranged to acquire one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and read relationship information between the one or more newly registered users and the one or more second objects; a second determining component arranged to, according to the one or more second objects having the followed relationship with the one or more newly registered user, determine a second followed set corresponding to the one or more newly registered users; and a second processing component arranged to match the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model; the device is further arranged to push recommended information to the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0015": "The device as claimed in claim 14 , wherein the device further comprises: a fourth acquiring component arranged to acquire recommended information; an extracting component arranged to extract one or more interest labels of the recommended information from content of the recommended information; a fifth acquiring component arranged to acquire historical behavior data of the one or more registered users, wherein the historical behavior data is used to record operational behavior of the one or more registered users for the recommended information; a third determining component arranged to determine one or more label weight values of the one or more interest labels according to the historical behavior data; and a fourth determining component arranged to determine the one or more interest label dictionaries corresponding to the one or more registered users according to the one or more label weight values.", + "c-en-0016": "The device as claimed in claim 15 , wherein the first processing component comprises: a first processing sub-component arranged to perform filtering in the first followed set to acquire a third followed set corresponding to the one or more registered users, wherein a method for performing filtering in the first followed set comprises at least one of a data filtering method, an index filtering method, a condition filtering method, and an information filtering method; a matching sub-component arranged to match the one or more registered users based on the third followed set to generate a registered user set, wherein the registered user set comprises the one or more registered users having the same third followed set; and a first generating component arranged to generate a user set label dictionary corresponding to the registered user set according to the one or more interest label dictionaries of the one or more registered users comprised in the registered user set.", + "c-en-0017": "The device as claimed in claim 16 , wherein the first generating component comprises: a first acquiring sub-component arranged to acquire a first user amount of the one or more registered users on the information client and a second user amount of the registered user set; a first calculating sub-component arranged to calculate a weight distribution average value of each of the interest labels according to the one or more label weight values and a first user amount; a second calculating sub-component arranged to calculate a set weight average value of each of the interest labels in the user set interest label dictionary according to the one or more label weight values and a second user amount of the one or more registered users in the registered user set; a third calculating sub-component arranged to calculate a registered user set weight value of the one or more interest labels in the user set interest label dictionary according to the weight distribution average value and the set weight average value; and a judging sub-component arranged to successively compare the registered user set weight value of the one or more interest labels in the user set interest label dictionary with a preset noise threshold; retaining the interest label corresponding to the registered user set weight value in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is greater than the preset noise threshold; and deleting the interest label corresponding to the registered user set weight value in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is less than or equal to the preset noise threshold.", + "c-en-0018": "The device as claimed in claim 17 , wherein the processing component comprises: a second processing sub-component arranged to perform filtering in the second followed set to acquire a fourth followed set corresponding to the one or more newly registered users, wherein a method for performing filtering in the first followed set comprises at least one of a data filtering method, an index filtering method, a condition filtering method, and an information filtering method; a first determining sub-component arranged to match the fourth followed set with the third followed set to determine the registered user set corresponding to the one or more newly registered users; and a second determining sub-component arranged to determine the one or more recommended interest labels of the one or more newly registered users according to the user set label dictionary of the registered user set corresponding to the one or more newly registered users.", + "c-en-0019": "The device as claimed in claim 17 , wherein the registered user set weight value V\u2032 of the one or more interest labels is determined by the following formula: V \u2032 = V / V base ; wherein V\u2032 denotes the registered user set weight value of an interest label i, V denotes the set weight average value of the interest label i, V base denotes the weight distribution average value of the interest label i.", + "c-en-0020": "The device as claimed in claim 15 , wherein the device further comprises: a pushing component arranged to push the recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "c-en-0021": "The device as claimed in claim 14 , wherein the device further comprises: a pushing component arranged to push the recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "p0001": "CROSS-REFERENCE TO RELATED PATENT APPLICATIONS", + "p0002": "The present application is a National Stage of International Patent Application No. PCT/CN2015/083804, filed Jul. 10, 2015, and claims the priority of Chinese Patent Application No. 201410645497.2, filed on Nov. 10, 2014, the disclosures of which are incorporated herein by reference in their entirety.", + "p0003": "TECHNICAL FIELD", + "p0004": "The embodiments of the disclosure relate to the computer field and, more particularly, to a method and device for social platform-based data mining.", + "p0005": "BACKGROUND", + "p0006": "At present, with the development of the computer technology and the gradual popularization of the Internet, more and more people acquire a variety of information through the Internet. And correspondingly, the amount of information on the Internet has become more abundant with the development of the computer technology and the popularity of the Internet.", + "p0007": "In recent years, with the rapid development of mobile Internet, people have gradually been accustomed to acquiring information content through the information client on the mobile terminal. In this way, the time that a user acquires information through the network becomes more fragmented. In this context, how to accurately provide users with valuable information that users are interested in becomes more important. In particular, it is an urgent problem to provide new users with valuable and interesting information.", + "p0008": "In the traditional technologies, the cold start problem of the recommendation system is a major challenge in the application of the products such as information clients. Herein the cold start problem of the recommendation system refers to the fact that the new user system lacks sufficient data for capturing effective recommended content that users are interested in. There is a widely used method in a number of solutions to the problem, that is, encouraging users to login the recommendation system with a Social Network Service account, for example: login with a social account such as microblog, Tencent QQ and Renren. The recommendation system can use the information of a user in a social platform to initialize the interest model of the user to make effective recommendation.", + "p0009": "On the one hand, there are still a lot of difficulties in using public data from a social platform for content recommendation in the practical application. For example, the published content of the social platform is often shorter and messy, the label content of a user is often unconventional , it is more difficult to understand by a machine learning algorithm and it is limited in helping improve the recommended service. For users who are not active on the SNS and have weak social relationships, public data on their SNS platforms is more limited in improving the recommendation effect. On the other hand, a mature content recommended service provider with a larger number of users often has accumulated a lot of user behavior information in the long-term operation process, such as: on-demand video and articles read or commented by the user. If this part of data may be effectively integrated and used with the public data of the SNS, it is possible to greatly improve the recommendation effect of the user. However, the existing technology basically focuses on the use of the public data provided by the SNS platform to mine a user interest model and recommend. It is difficult to achieve this method, and the accuracy is low.", + "p0010": "There is no effective solution for the problem in the traditional art that targeted information cannot be provided because a newly registered user has no historical browsing record.", + "p0011": "SUMMARY", + "p0012": "The main object of the embodiments of the disclosure is to provide a method and device for social platform-based data mining for solving the problem in the traditional art that targeted information cannot be provided because a newly registered user has no historical browsing record.", + "p0013": "In order to achieve the above object, according to an aspect of an embodiment of the disclosure, there is provided a method for social platform-based data mining. The method includes: acquiring one or more interest label dictionaries of one or more registered users on an information client; acquiring one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and reading the relationship information between the one or more registered users and the one or more first objects; according to the one or more first objects having the followed relationship with the one or more registered users, determining one or more first followed sets corresponding to the one or more registered users; according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, constructing an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; acquiring one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and reading relationship information between the one or more newly registered users and the one or more second objects; according to the one or more second objects having the followed relationship with the one or more newly registered user, determining a second followed set corresponding to the one or more newly registered users; and matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0014": "In order to achieve the above object, according to another aspect of an embodiment of the disclosure, there is provided a device for social platform-based data mining. The device includes: a first acquiring component arranged to acquire one or more interest label dictionaries of one or more registered users on an information client; a second acquiring component arranged to acquire one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and read the relationship information between the one or more registered users and the one or more first objects; a first determining component arranged to, according to the one or more first objects having the followed relationship with the one or more registered users, determine one or more first followed sets corresponding to the one or more registered users; a first processing component arranged to, according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, construct an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; a third acquiring component arranged to acquire one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and read relationship information between the one or more newly registered users and the one or more second objects; a second determining component arranged to, according to the one or more second objects having the followed relationship with the one or more newly registered user, determine a second followed set corresponding to the one or more newly registered users; and a second processing component arranged to match the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0015": "According to an embodiment of the disclosure, by acquiring one or more interest label dictionaries of one or more registered users on an information client; acquiring one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and reading the relationship information between the one or more registered users and the one or more first objects; according to the one or more first objects having the followed relationship with the one or more registered users, determining one or more first followed sets corresponding to the one or more registered users; according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, constructing an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; acquiring one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and reading relationship information between the one or more newly registered users and the one or more second objects; according to the one or more second objects having the followed relationship with the one or more newly registered user, determining a second followed set corresponding to the one or more newly registered users; and matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model, the problem in the traditional art is solved that targeted information cannot be provided because a newly registered user has no historical browsing record, and the effect is achieved for providing targeted information for users through the followed relationship of the newly registered users in the social platform.", + "p0016": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0017": "The drawings forming a part of the embodiments of the disclosure are described here to provide further understanding of the embodiments of the disclosure. The schematic embodiments and description of the disclosure are adopted to explain the embodiments of the disclosure, and do not form improper limits to the embodiments of the disclosure. In the drawings:", + "p0018": "FIG. 1 is a flow diagram of a social platform-based data mining according to a first embodiment of the disclosure;", + "p0019": "FIG. 2 is a flow diagram of a preferred social platform-based data mining according to a first embodiment of the disclosure;", + "p0020": "FIG. 3 is a flow schematic diagram of matching the registered users through the followed set in the microblog to generate a registered user set;", + "p0021": "FIG. 4 is a structure schematic diagram of a device for social platform-based data mining according to a second embodiment of the disclosure;", + "p0022": "FIG. 5 is a structure schematic diagram of a preferred device for social platform-based data mining according to a second embodiment of the disclosure; and", + "p0023": "FIG. 6 is a structure schematic diagram of a preferred device for social platform-based data mining according to a second embodiment of the disclosure.", + "p0024": "DETAILED DESCRIPTION OF THE EMBODIMENTS", + "p0025": "It is to be noted that the embodiments of the disclosure and the characteristics in the embodiments may be combined with each other under the condition of no conflicts. The embodiments of the disclosure are described below with reference to the drawings and the embodiments in detail.", + "p0026": "In order to make a better understanding of the solution of the embodiments of the disclosure by those skilled in the art, the technical solutions in the embodiments of the disclosure will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the disclosure, and it will be apparent that the described embodiments are merely a part of the embodiments of the disclosure rather than all embodiments. All other embodiments acquired by those of ordinary skill in the art without making creative work are intended to be within the scope of protection of the embodiments of the disclosure, based on embodiments of the disclosure.", + "p0027": "It is to be noted that the terms such as \u201ca first\u201d and \u201ca second\u201d in the specification and claims of the embodiments of the disclosure and in the above accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It is to be understood that the data thus used are interchangeable where appropriate to enable the embodiments of the disclosure to be described herein. In addition, the terms such as \u201cincluding\u201d, \u201chaving\u201d or any other variants thereof are intended to cover a non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those explicitly listed steps or units, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products, or devices.", + "p0028": "Embodiment 1", + "p0029": "The embodiment of the disclosure provides a method for social platform-based data mining.", + "p0030": "FIG. 1 is a flow diagram of a social platform-based data mining according to a first embodiment of the disclosure. As shown in FIG. 1 , the method includes the following steps.", + "p0031": "Step S 11 : One or more interest label dictionaries of one or more registered users on an information client are acquired.", + "p0032": "In the above Step S 11 of the disclosure, the one or more interest label dictionaries corresponding to each registered user is analyzed and acquired by collecting the history browsing behavior of the one or more registered users.", + "p0033": "Step S 13 : One or more first objects having followed relationship with the one or more registered users on the information client in a social platform are acquired and the relationship information between the one or more registered users and the one or more first objects are read.", + "p0034": "In the above Step S 13 of the disclosure, the object having followed relationship with the one or more registered users is determined by reading the followed relationship information of the one or more registered users on the social platform.", + "p0035": "In the practical applications, the followed relationship may be a friend relationship in Tencent QQ software, or followed relationship in the microblog, or a friend relationship in Renren.", + "p0036": "Step S 15 : One or more first followed sets corresponding to the one or more registered users are determined according to the one or more first objects having the followed relationship with the one or more registered users.", + "p0037": "In the above Step S 15 of the disclosure, a first followed set of each registered user is determined by arranging the one or more first objects having the followed relationship with each registered user.", + "p0038": "Step S 17 : An interest model is constructed according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label.", + "p0039": "In the above Step S 17 of the disclosure, the registered users having different first followed sets are classified into registered user sets corresponding to several first followed sets by analyzing the followed set of each registered user, and a user set label dictionary corresponding to the first followed set is generated through the one or more interest label dictionaries of the registered users in the registered user set so as to determine a corresponding relationship between the first followed set and an interest label.", + "p0040": "Step S 19 : One or more second objects having followed relationship with one or more newly registered users on the information client in the social platform are acquired, and the relationship information between the one or more newly registered users and the one or more second objects is read.", + "p0041": "In the above Step S 19 of the disclosure, the one or more second objects having followed relationship with one or more newly registered users is determined by reading the followed relationship information of the newly registered user on the social platform.", + "p0042": "In the practical applications, the followed relationship may be a friend relationship in Tencent QQ software, or followed relationship in the microblog, or a friend relationship in Renren.", + "p0043": "Step S 21 : A second followed set corresponding to the one or more newly registered users is determined according to the one or more second objects having the followed relationship with the newly registered user.", + "p0044": "In the above Step S 21 of the disclosure, a second followed set corresponding to the one or more newly registered users is determined by arranging the one or more second objects having the followed relationship with the newly registered user.", + "p0045": "Step S 23 : The second followed set is matched with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0046": "In the above step S 23 of the disclosure, a first followed set that is matched with the second followed set corresponding to the one or more newly registered users is acquired by matching the second followed set corresponding to the one or more newly registered users with several first followed sets in the interest model so as to determine the interest label of the newly registered user through the first followed set.", + "p0047": "Specifically, through Step S 11 to Step S 23 , the registered users having the same first followed set in the social platform are grouped to acquire the registered user set corresponding to the one or more first followed sets, and the user set label dictionary corresponding to the registered user set is acquired by acquiring the one or more interest label dictionaries of the registered users on the information client. In this way, an interest model with a corresponding relationship between the first followed set and the user set label dictionary is constructed. After acquiring the second followed set corresponding to the one or more newly registered users, the recommendation interest label of the newly registered user may be acquired by matching the second followed set directly with the first followed set in the interest model.", + "p0048": "In the practical applications, it is generally believed that the relationship in the social platform reflects the interest similarity of users. Based on different assumptions, we can use different methods to find other users with similar interests to a user in the social platform. Different assumptions apply to different types of social platforms. For example, for Tencent QQ and WeChat, as social platforms which emphasize two-way communication, it may be assumed that the interests between friends are similar. For microblog, as a social platform which emphasizes one-way focus, it may be assumed that users having common followed objects are similar in interest. For example, if two users follow Lei Jun and Huang Zhang, they are likely to be interested in smart phones.", + "p0049": "Taking the microblog as an example, the social platform filters the content of the followed lists of the microblog of the registered users on the information client, and filters the followed object in which the number of fans is more than a certain value or filters the followed object in which the number of fans is top-ranked to constitute a first followed set. The followed lists of the microblog of all the registered users are filtered in the same way of filtering to acquire one or more first followed sets corresponding to each registered user. The registered users having the same first followed set are classified as several registered user sets, and each registered user set has a different first followed set. The user set label dictionary corresponding to each registered user set is acquired by collecting the one or more interest label dictionaries of the one or more registered users in the registered user set. After a newly registered user registers the information client and authorizes the information client to call the microblog public data, the followed lists of the new user are also filtered in the same way. The filtered the second followed set is matched with the first followed set of several registered users to determine the registered user set to which the new user belongs and to acquire the user set label dictionary corresponding to the registered user set, that is, the one or more recommended interest labels of the one or more newly registered users.", + "p0050": "In view of the above, the embodiments of the disclosure solve the problem in the traditional art that targeted information cannot be provided because a newly registered user has no historical browsing record. The effect is achieved for providing targeted information for users through the followed relationship of the newly registered users in the social platform.", + "p0051": "Preferably, as shown in FIG. 2 , in the preferred embodiment provided by the disclosure, before acquiring the one or more interest label dictionaries of the one or more registered users on the information client in Step S 11 , the method includes the following steps.", + "p0052": "Step S 101 : Recommended information is acquired.", + "p0053": "Step S 103 : The interest label of the recommended information is extracted from content of the recommended information.", + "p0054": "Step S 105 : Historical behavior data of the one or more registered users is acquired, wherein the historical behavior data is used to record operational behavior of the one or more registered users for the recommended information.", + "p0055": "Step S 107 : One or more label weight values of the one or more interest labels are determined according to the historical behavior data.", + "p0056": "Step S 109 : The one or more interest label dictionaries corresponding to the one or more registered users are determined according to the one or more label weight values.", + "p0057": "Specifically, the contents of all the recommended information in the information client are analyzed through Step S 101 to Step S 109 , and the interest label is extracted for each piece of recommended information according to the contents of the recommended information. When the one or more registered users operates the recommended information, the operation behavior of the one or more registered users is recorded. The interest label corresponding to the recommended information is subjected to weighted calculation according to the operation behavior of the recommended information, and the weight value of the one or more interest labels corresponding to the one or more registered users is acquired by calculation. When the label weight value is greater than the threshold, the label is added to the one or more interest label dictionaries corresponding to that user.", + "p0058": "In the practical applications, the recommended service in the information client will provide an interest label for the recommended content in the information client, for example: the classification of content: science and technology, football, basketball, etc.; the classification of the corresponding people: technical indoorsmen, outdoor enthusiasts, adolescents, etc.; keywords for content: iPhone, tank contest, Bayern Munich and so on. These interest labels are sometimes artificially edited, and are sometimes identified by the algorithm automatically analyzing the recommended information.", + "p0059": "In the case that all the recommended information that the recommended service may recommend has interest labels, the one or more interest label dictionaries of the user by recording the behavior data of the one or more registered users using the recommended service, such as browsing content, click/collection/comment content, etc., and according to the interest label corresponding to the information content. This interest label dictionary describes what interest labels the user has, and how much is the weight of each interest label. This interest label dictionary may be used as an interest model in subsequent steps.", + "p0060": "Specifically, the calculation method of the label weight value of the one or more interest labels may include the following content.", + "p0061": "First of all, a weight w is set for each user act, such as keeping 1 score for click, keeping \u22120.2 score for browse without click, and keeping 5 scores for collection.", + "p0062": "Given a user act sequence , the interest label weight value of the user is calculated as follows:", + "p0063": "V=\u03a3 i Ti\u00b7wi", + "p0064": "wherein Ti denotes the interest label vector of the i-th user act, and wi denotes the weight of the i-th user act.", + "p0065": "In an example embodiment provided by the disclosure, the step of constructing the interest model according to the one or more interest label dictionaries of the one or more registered users and the first followed set in Step S 17 includes the following steps.", + "p0066": "Step S 171 : The first followed set is filtered to acquire a third followed set corresponding to the one or more registered users, wherein the filtering method at least includes a data filtering method, an index filtering method, a condition filtering method, and an information filtering method.", + "p0067": "Step S 173 : The one or more registered users is matched based on the third followed set to generate a registered user set, wherein the registered user set includes the one or more registered users having the same third followed set.", + "p0068": "Step S 175 : A user set label dictionary corresponding to the registered user set is generated according to the one or more interest label dictionaries of the one or more registered users included in the registered user set.", + "p0069": "Specifically, the first followed set of the one or more registered users is first filtered through Step S 171 to Step S 175 . The first followed set may be filtered according to the number of followers and/or the number of friends and/or the degree of activity and other conditions. The users who are not active and have fewer friends are removed from the first followed set to generate a filtered third followed set.", + "p0070": "The filtered registered users are matched through the third followed set. The registered users in which the matching degree of the third followed set is greater than the pre-set threshold or the third followed set is the same are classified into the same registered user set. According to the content difference of the third followed set, there may be many registered user sets. Of course, the third followed set can also be defined artificially, and the third followed set defined artificially groups the registered users into different registered user sets.", + "p0071": "The user set label dictionary corresponding to the currently registered user set is generated according to the content of the one or more interest label dictionaries corresponding to each registered user among the registered user sets.", + "p0072": "The above social platform takes the microblog as an example. As shown in FIG. 3 , FIG. 3 is a flow schematic diagram of matching the registered users through the followed set in the microblog to generate a registered user set.", + "p0073": "The users with fewer fans in the followed list are filtered by acquiring the followed list of registered users and taking the number of fans as a filtering condition. A third followed set is generated according to the filtered followed list. Of course, for the microblog, the third followed set can also be artificially defined. For example, the specific users in the microblog are classified in accordance with the user category. The users in the field of computer internet such as Li Kaifu, Lei Jun, Zhou Hongyi, and Li Yanhong may be classified into a third followed set, the users in the field of entertainment media such as He Jiong, Xie Na, Dai Jun may be classified into a third followed set, and the users in the field of sports such as Wei Kexing, Li Na, Liu Xiang may be classified into a third followed set.", + "p0074": "According to the third followed set, the registered users are classified and the registered users who have a common third followed set are divided into a registered user set to achieve the purpose of user groups of similar interest.", + "p0075": "In an example embodiment provided by the disclosure, the step of generating a user set label dictionary corresponding to the registered user set according to the one or more interest label dictionaries of the one or more registered users included in the registered user set in Step S 175 includes the following steps.", + "p0076": "Step S 1751 : A first user amount of the registered users on the information client and a second user amount of the registered user set are acquired.", + "p0077": "Step S 1753 : A weight distribution average value of each of the interest labels is calculated according to the one or more label weight values and a first user amount.", + "p0078": "Step S 1755 : A set weight average value of each of the interest labels in the user set interest label dictionary is calculated according to the one or more label weight values and a second user amount of the one or more registered users in the registered user set.", + "p0079": "Step S 1757 : A registered user set weight value of the one or more interest labels in the user set interest label dictionary is calculated according to the weight distribution average value and the set weight average value.", + "p0080": "Step S 1759 : The registered user set weight value of the one or more interest labels in the user set interest label dictionary is successively compared with a preset noise threshold.", + "p0081": "The interest label corresponding to the registered user set weight value is retained in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is greater than the preset noise threshold.", + "p0082": "The interest label corresponding to the registered user set weight value is deleted in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is less than or equal to the preset noise threshold.", + "p0083": "Specifically, in the practical applications of Step S 1751 to Step S 1759 , the social platform takes the microblog as an example. After finding a user group of similar interest, the interest label dictionarys of these user individuals may be combined to acquire a group interest model. The easiest way is to add the user label vectors directly. But in the practical applications, it is found that this way will result in a lot of noise because there are many followers for big microblog IDs in some fields, and many people just follow since the big ID is famous so that the following behavior itself cannot reflect their own interests. If the interest label vectors of these users are simply added, the meaningful signal is easily flooded with general interest. In the case of the actual experiment, it is found by analyzing the microblog users who follow Wang Xing that the maximum weight interest labels are not \u201cInternet\u201d or \u201cO2O\u201d, but are \u201centertainment\u201d and \u201csocial news\u201d. This is because \u201centertainment\u201d and \u201csocial news\u201d are common interest labels. Many users with these two labels follow Wang Xing because he is the founder of Meituan, but do not follow \u201cInternet\u201d and \u201cO2O\u201d so much in fact. In the end, if we consider all these users indiscriminately, the result will be acquired that the weight of \u201centertainment\u201d and \u201csocial news\u201d is higher than that of \u201cInternet\u201d and \u201cO2O\u201d.", + "p0084": "How to remove background noise is the core technology of effectively mining the group interest. In practice, we first need to count the weight distribution average value of the registered users in all the websites:", + "p0086": "where N denotes the number of all registered users, and V n denotes the interest label weight distribution of a user.", + "p0087": "By the above formula, the weight distribution average value V base of all the users on the interest label i is further acquired.", + "p0088": "Then for the registered user set having a same condition in the followed relationship , the interest label vector V is given to the registered user set group so as to acquire the registered user set weight value V\u2032 for removing the noise, respectively:", + "p0089": "V \u2032 = V / V base ;", + "p0090": "where V\u2032 denotes the registered user set weight value of the interest label i, V denotes the set weight average value of the interest label i, V base denotes the weight distribution average value of all the users on the interest label i.", + "p0091": "By comparing the registered user set weight value V\u2032 with a preset noise threshold, when the registered user set weight value V\u2032 is smaller than the noise threshold, it is proved that the interest label is a noise label, and should be removed from the current user set label dictionary; and when the registered user set weight value V\u2032 is equal to or greater than the noise threshold, it is judged that the interest label is a non-noise label and the label is retained in the current user set label dictionary.", + "p0092": "Preferably, in the preferred embodiment provided by the disclosure, the step of matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model in Step S 23 includes the following steps.", + "p0093": "Step S 231 : The second followed set is filtered to acquire a fourth followed set corresponding to the one or more newly registered users, wherein the filtering method at least includes a data filtering method, an index filtering method, a condition filtering method, and an information filtering method.", + "p0094": "Step S 233 : The fourth followed set is matched with the third followed set to determine the registered user set corresponding to the one or more newly registered users.", + "p0095": "Step S 235 : The one or more recommended interest labels of the one or more newly registered users is determined according to the user set label dictionary of the registered user set corresponding to the one or more newly registered users.", + "p0096": "Specifically, the second followed set of the one or more registered users is first filtered through Step S 231 to Step S 235 . The second followed set may be filtered according to the number of followers and/or the number of friends and/or the degree of activity and other conditions. The users who are not active and have fewer friends are removed from the second followed set to generate a filtered fourth followed set. Herein the filtering method may be the same as the filtering method used in Sep 171 , and other filtering methods may also be used. The filtering method used is not limited as long as the purpose of optimizing the second followed set may be achieved.", + "p0097": "Then the fourth followed set is matched with each of the third followed sets. When the matching degree of the fourth followed set corresponding to the one or more newly registered users and the third followed set is greater than the preset threshold or the third followed set is exactly the same, it is determined that the newly registered user is matched with the third followed set so as to determine the registered user set to which the newly registered user belongs.", + "p0098": "The recommendation label recommended for the new user is determined according to the user set label dictionary of the registered user set to which the newly registered user belongs.", + "p0099": "In the practical applications, after mining a group interest model of a user group having similar interest to that of the newly registered users, we can integrate the group interest model and the user individual interest model according to a certain weight, and then recommend the content according to the integrated interest model. Specifically, given an integrated interest model , we can recommend some of the highest quality content under the label according to the ratio such as the weight of each interest label.", + "p0100": "It is to be noted that, for a new user, we cannot acquire the individual interest model of the new user without any of the act data of the user in the station. But if the new user logs in the information client with a network account of the social platform, we can acquire the social relations of the newly registered user on the social platform. The targeted recommended information may be achieved by mining the user group having similar interest within the station and using this group interest model to recommend content to users. In practice, this practice has a better effect than randomly recommending or recommending the most popular content.", + "p0101": "Preferably, in the preferred embodiment provided by the disclosure, after matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model in Step S 23 , the method further includes the following step.", + "p0102": "Step S 24 : The recommended information is pushed for the one or more newly registered users according to the one or more recommendation interest labels.", + "p0103": "Specifically, through Step S 24 , the recommended information matching the interest label is pushed for the newly registered user according to the interest label determined for the newly registered user through the above steps.", + "p0104": "As may be seen from the technical solution, the embodiments of the disclosure effectively combine the SNS public data and the recommended service private data together as the user recommended content. Compared with using only the SNS public data or the recommended service private data, the integration of the two data is conducive to more accurately recommending personalized content. Moreover, the integrating method proposed by the embodiments of the disclosure can also utilize the integration of the two data , which is also the effect that the traditional method cannot achieve.", + "p0105": "One feature of the embodiments of the disclosure is that if the recommended service provider has more users, the effect of this method is the better. Because the user group of such a service provider has a relatively large coverage for user groups of the SNS. The situation will not occur that a use randomly gives a social account and most of his/her friends or fans are not users in the station so that the group interest cannot be mined. This is a significant competitive advantage for products such as headlines today that have 100 million users, but it is a technical barrier for some smaller recommended products.", + "p0106": "Embodiment 2", + "p0107": "The embodiment of the disclosure further provides a device for social platform-based data mining. As shown in FIG. 4 , the device includes a first acquiring component 30 , a second acquiring component 32 , a first determining component 34 , a first processing component 36 , a third acquiring component 38 , a second determining component 40 , and a second processing component 42 .", + "p0108": "Herein the first acquiring component 30 is arranged to acquire one or more interest label dictionaries of one or more registered users on an information client.", + "p0109": "The first acquiring component 30 of the disclosure is arranged to analyze and acquire the one or more interest label dictionaries corresponding to each registered user by collecting the history browsing behavior of the one or more registered users.", + "p0110": "The second acquiring component 32 is arranged to acquire one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and read the relationship information between the one or more registered users and the one or more first objects.", + "p0111": "The second acquiring component 32 of the disclosure is arranged to determine the object having followed relationship with the one or more registered users by reading the followed relationship information of the one or more registered users on the social platform.", + "p0112": "In the practical applications, the followed relationship may be a friend relationship in Tencent QQ software, or followed relationship in the microblog, or a friend relationship in Renren.", + "p0113": "The first determining component 34 is arranged to determine one or more first followed sets corresponding to the one or more registered users according to the one or more first objects having the followed relationship with the one or more registered users.", + "p0114": "The first determining component 34 of the disclosure is arranged to determine a first followed set of each registered user arranging the one or more first objects having the followed relationship with each registered user.", + "p0115": "The first processing component 36 is arranged to construct an interest model according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label.", + "p0116": "The first processing component 36 of the disclosure is arranged to classify the registered users having different first followed sets into registered user sets corresponding to several first followed sets by analyzing the followed set of each registered user, and generate a user set label dictionary corresponding to the first followed set through the one or more interest label dictionaries of the registered users in the registered user set so as to determine a corresponding relationship between the first followed set and an interest label.", + "p0117": "The third acquiring component 38 is arranged to acquire one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and read the relationship information between the one or more newly registered users and the one or more second objects.", + "p0118": "The third acquiring component 38 of the disclosure is arranged to determine the one or more second objects having followed relationship with one or more newly registered users by reading the followed relationship information of the newly registered user on the social platform.", + "p0119": "In the practical applications, the followed relationship may be a friend relationship in Tencent QQ software, or followed relationship in the microblog, or a friend relationship in Renren.", + "p0120": "The second determining component 40 is arranged to determine a second followed set corresponding to the one or more newly registered users according to the one or more second objects having the followed relationship with the newly registered user.", + "p0121": "The second determining component 40 of the disclosure is arranged to determine a second followed set corresponding to the one or more newly registered users by arranging the one or more second objects having the followed relationship with the newly registered user.", + "p0122": "The second processing component 42 is arranged to match the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0123": "The second processing component 42 of the disclosure is arranged to acquire a first followed set that is matched with the second followed set corresponding to the one or more newly registered users by matching the second followed set corresponding to the one or more newly registered users with several first followed sets in the interest model so as to determine the interest label of the newly registered user through the first followed set.", + "p0124": "Specifically, through the first acquiring component 30 , the second acquiring component 32 , the first determining component 34 , the first processing component 36 , the third acquiring component 38 , the second determining component 40 , and the second processing component 42 , the registered users having the same first followed set in the social platform are grouped to acquire the registered user set corresponding to the one or more first followed sets, and the user set label dictionary corresponding to the registered user set is acquired by acquiring the one or more interest label dictionaries of the registered users on the information client. In this way, an interest model with a corresponding relationship between the first followed set and the user set label dictionary is constructed. After acquiring the second followed set corresponding to the one or more newly registered users, the recommendation interest label of the newly registered user may be acquired by matching the second followed set directly with the first followed set in the interest model.", + "p0125": "In the practical applications, it is generally believed that the relationship in the social platform reflects the interest similarity of users. Based on different assumptions, we can use different methods to find other users with similar interests to a user in the social platform. Different assumptions apply to different types of social platforms. For example, for Tencent QQ and WeChat, as social platforms which emphasize two-way communication, it may be assumed that the interests between friends are similar. For microblog, as a social platform which emphasizes one-way focus, it may be assumed that users having common followed objects are similar in interest. For example, if two users follow Lei Jun and Huang Zhang, they are likely to be interested in smart phones.", + "p0126": "Taking the microblog as an example, the social platform filters the content of the followed lists of the microblog of the registered users on the information client, and filters the followed object in which the number of fans is more than a certain value or filters the followed object in which the number of fans is top-ranked to constitute a first followed set. The followed lists of the microblog of all the registered users are filtered in the same way of filtering to acquire one or more first followed sets corresponding to each registered user. The registered users having the same first followed set are classified as several registered user sets, and each registered user set has a different first followed set. The user set label dictionary corresponding to each registered user set is acquired by collecting the one or more interest label dictionaries of the one or more registered users in the registered user set. After a newly registered user registers the information client and authorizes the information client to call the microblog public data, the followed lists of the new user are also filtered in the same way. The filtered the second followed set is matched with the first followed set of several registered users to determine the registered user set to which the new user belongs and to acquire the user set label dictionary corresponding to the registered user set, that is, the one or more recommended interest labels of the one or more newly registered users.", + "p0127": "In view of the above, the embodiments of the disclosure solve the problem in the traditional art that targeted information cannot be provided because a newly registered user has no historical browsing record. The effect is achieved for providing targeted information for users through the followed relationship of the newly registered users in the social platform.", + "p0128": "Preferably, in the preferred embodiment provided by the disclosure, as shown in FIG. 5 , the device further includes a fourth acquiring component 281 , an extracting component 283 , a fifth acquiring component 285 , a third determining component 287 , and a fourth determining component 289 .", + "p0129": "Herein the fourth acquiring component 281 is arranged to acquire recommended information.", + "p0130": "The extracting component 283 is arranged to extract one or more interest labels of the recommended information from content of the recommended information.", + "p0131": "The fifth acquiring component 285 is arranged to acquire historical behavior data of the one or more registered users, wherein the historical behavior data is used to record operational behavior of the one or more registered users for the recommended information.", + "p0132": "The third determining component 287 is arranged to determine one or more label weight values of the one or more interest labels according to the historical behavior data.", + "p0133": "The fourth determining component 289 is arranged to determine the one or more interest label dictionaries corresponding to the one or more registered users according to the one or more label weight values.", + "p0134": "Specifically, through the fourth acquiring component 281 , the extracting component 283 , the fifth acquiring component 285 , the third determining component 287 , and the fourth determining component 289 , the contents of all the recommended information in the information client are analyzed, and the interest label is extracted for each piece of recommended information according to the contents of the recommended information. When the one or more registered users operates the recommended information, the operation behavior of the one or more registered users is recorded. The interest label corresponding to the recommended information is subjected to weighted calculation according to the operation behavior of the recommended information, and the weight value of the one or more interest labels corresponding to the one or more registered users is acquired by calculation. When the label weight value is greater than the threshold, the label is added to the one or more interest label dictionaries corresponding to that user.", + "p0135": "In the practical applications, the recommended service in the information client will provide an interest label for the recommended content in the information client, for example: the classification of content: science and technology, football, basketball, etc.; the classification of the corresponding people: technical indoorsmen, outdoor enthusiasts, adolescents, etc.; keywords for content: iPhone, tank contest, Bayern Munich and so on. These interest labels are sometimes artificially edited, and are sometimes identified by the algorithm automatically analyzing the recommended information.", + "p0136": "In the case that all the recommended information that the recommended service may recommend has interest labels, the one or more interest label dictionaries of the user by recording the behavior data of the one or more registered users using the recommended service, such as browsing content, click/collection/comment content, etc., and according to the interest label corresponding to the information content. This interest label dictionary describes what interest labels the user has, and how much is the weight of each interest label. This interest label dictionary may be used as an interest model in subsequent steps.", + "p0137": "Specifically, the calculation method of the label weight value of the one or more interest labels may include the following content.", + "p0138": "First of all, a weight w is set for each user act, such as keeping 1 score for click, keeping \u22120.2 score for browse without click, and keeping 5 scores for collection.", + "p0139": "Given a user act sequence , the interest label weight value of the user is calculated as follows:", + "p0140": "V=\u03a3 i Ti\u00b7wi", + "p0141": "where Ti denotes the interest label vector of the i-th user act, and wi denotes the weight of the i-th user act.", + "p0142": "Preferably, in the preferred embodiment provided by the disclosure, the first processing component 36 includes a first sub-processing component 361 , a sub-matching component 363 , and a first generating component 365 .", + "p0143": "Herein the first sub-processing component 361 is arranged to perform filtering in the first followed set to acquire a third followed set corresponding to the one or more registered users, wherein the filtering method at least includes a data filtering method, an index filtering method, a condition filtering method, and an information filtering method.", + "p0144": "The sub-matching component 363 is arranged to match the one or more registered users based on the third followed set to generate a registered user set, wherein the registered user set includes the one or more registered users having the same third followed set.", + "p0145": "The first generating component 365 is arranged to generate a user set label dictionary corresponding to the registered user set according to the one or more interest label dictionaries of the one or more registered users included in the registered user set.", + "p0146": "Specifically, through the first sub-processing component 361 , the sub-matching component 363 , and the first generating component 365 , the first followed set of the one or more registered users is first filtered. The first followed set may be filtered according to the number of followers and/or the number of friends and/or the degree of activity and other conditions. The users who are not active and have fewer friends are removed from the first followed set to generate a filtered third followed set.", + "p0147": "The filtered registered users are matched through the third followed set. The registered users in which the matching degree of the third followed set is greater than the pre-set threshold or the third followed set is the same are classified into the same registered user set. According to the content difference of the third followed set, there may be many registered user sets. Of course, the third followed set can also be defined artificially, and the third followed set defined artificially groups the registered users into different registered user sets.", + "p0148": "The user set label dictionary corresponding to the currently registered user set is generated according to the content of the one or more interest label dictionaries corresponding to each registered user among the registered user sets.", + "p0149": "The above social platform takes the microblog as an example. As shown in FIG. 3 , FIG. 3 is a flow schematic diagram of matching the registered users through the followed set in the microblog to generate a registered user set.", + "p0150": "The users with fewer fans in the followed list are filtered by acquiring the followed list of registered users and taking the number of fans as a filtering condition. A third followed set is generated according to the filtered followed list. Of course, for the microblog, the third followed set can also be artificially defined. For example, the specific users in the microblog are classified in accordance with the user category. The users in the field of computer internet such as Li Kaifu, Lei Jun, Zhou Hongyi, and Li Yanhong may be classified into a third followed set, the users in the field of entertainment media such as He Jiong, Xie Na, Dai Jun may be classified into a third followed set, and the users in the field of sports such as Wei Kexing, Li Na, Liu Xiang may be classified into a third followed set.", + "p0151": "According to the third followed set, the registered users are classified and the registered users who have a common third followed set are divided into a registered user set to achieve the purpose of user groups of similar interest.", + "p0152": "In an example embodiment provided by the disclosure, the first generating component 365 includes a first acquiring sub-component 3651 , a first calculating sub-component 3652 , a second calculating sub-component 3653 , a third calculating sub-component 3654 , and a judging sub-component 3655 .", + "p0153": "Herein the first acquiring sub-component 3651 is arranged to acquire a first user amount of the registered users on the information client and a second user amount of the registered user set.", + "p0154": "The first calculating sub-component 3652 is arranged to calculate a weight distribution average value of each of the interest labels according to the one or more label weight values and a first user amount.", + "p0155": "The second calculating sub-component 3653 is arranged to calculate a set weight average value of each of the interest labels in the user set interest label dictionary according to the one or more label weight values and a second user amount of the one or more registered users in the registered user set.", + "p0156": "The third calculating sub-component 3654 is arranged to calculate a registered user set weight value of the one or more interest labels in the user set interest label dictionary according to the weight distribution average value and the set weight average value.", + "p0157": "The judging sub-component 3655 is arranged to successively compare the registered user set weight value of the one or more interest labels in the user set interest label dictionary with a preset noise threshold.", + "p0158": "The interest label corresponding to the registered user set weight value is retained in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is greater than the preset noise threshold.", + "p0159": "The interest label corresponding to the registered user set weight value is deleted in the user set label dictionary when the registered user set weight value of the one or more interest labels in the user set interest label dictionary is less than or equal to the preset noise threshold.", + "p0160": "Specifically, through the first acquiring sub-component 3651 , the first calculating sub-component 3652 , the second calculating sub-component 3653 , the third calculating sub-component 3654 , and the judging sub-component 3655 , in the practical applications, the social platform takes the microblog as an example. After finding a user group of similar interest, the interest label dictionarys of these user individuals may be combined to acquire a group interest model. The easiest way is to add the user label vectors directly. But in the practical applications, it is found that this way will result in a lot of noise because there are many followers for big microblog IDs in some fields, and many people just follow since the big ID is famous so that the following behavior itself cannot reflect their own interests. If the interest label vectors of these users are simply added, the meaningful signal is easily flooded with general interest. In the case of the actual experiment, it is found by analyzing the microblog users who follow Wang Xing that the maximum weight interest labels are not \u201cInternet\u201d or \u201cO2O\u201d, but are \u201centertainment\u201d and \u201csocial news\u201d. This is because \u201centertainment\u201d and \u201csocial news\u201d are common interest labels. Many users with these two labels follow Wang Xing because he is the founder of Meituan, but do not follow \u201cInternet\u201d and \u201cO2O\u201d so much in fact. In the end, if we consider all these users indiscriminately, the result will be acquired that the weight of \u201centertainment\u201d and \u201csocial news\u201d is higher than that of \u201cInternet\u201d and \u201cO2O\u201d.", + "p0161": "How to remove background noise is the core technology of effectively mining the group interest. In practice, we first need to count the weight distribution average value of the registered users in all the websites:", + "p0163": "where N denotes the number of all registered users, and V n denotes the interest label weight distribution of a user.", + "p0164": "By the above formula, the weight distribution average value V base of all the users on the interest label i is further acquired.", + "p0165": "Then for the registered user set having a same condition in the followed relationship , the interest label vector V is given to the registered user set group so as to acquire the registered user set weight value V\u2032 for removing the noise, respectively:", + "p0166": "V \u2032 = V / V base ;", + "p0167": "where V\u2032 denotes the registered user set weight value of the interest label i, V denotes the set weight average value of the interest label i, V base denotes the weight distribution average value of all the users on the interest label i.", + "p0168": "By comparing the registered user set weight value V\u2032 with a preset noise threshold, when the registered user set weight value V\u2032 is smaller than the noise threshold, it is proved that the interest label is a noise label, and should be removed from the current user set label dictionary; and when the registered user set weight value V\u2032 is equal to or greater than the noise threshold, it is judged that the interest label is a non-noise label and the label is retained in the current user set label dictionary.", + "p0169": "By comparing the registered user set weight value V\u2032 with a preset noise threshold, when the registered user set weight value V\u2032 is smaller than the noise threshold, it is proved that the interest label is a noise label, and should be removed from the current user set label dictionary; and when the registered user set weight value V\u2032 is equal to or greater than the noise threshold, it is judged that the interest label is a non-noise label and the label is retained in the current user set label dictionary.", + "p0170": "Preferably, in the preferred embodiment provided by the disclosure, the second processing component 42 includes a second processing sub-component 421 , a first determining sub-component 423 , and a second determining sub-component 425 .", + "p0171": "Herein the second processing sub-component 421 is arranged to perform filtering in the second followed set to acquire a fourth followed set corresponding to the one or more newly registered users, wherein the filtering device at least includes a data filtering method, an index filtering method, a condition filtering method, and an information filtering method.", + "p0172": "The first determining sub-component 423 is arranged to match the fourth followed set with the third followed set to determine the registered user set corresponding to the one or more newly registered users.", + "p0173": "The second determining sub-component 425 is arranged to determine the one or more recommended interest labels of the one or more newly registered users according to the user set label dictionary of the registered user set corresponding to the one or more newly registered users.", + "p0174": "Specifically, through the second processing sub-component 421 , the first determining sub-component 423 , and the second determining sub-component 425 , the second followed set of the registered user is first filtered. The second followed set may be filtered according to the number of followers and/or the number of friends and/or the degree of activity and other conditions. The users who are not active and have fewer friends are removed from the second followed set to generate a filtered fourth followed set. Herein the filtering method may be the same as the filtering method used in Sep 171 , and other filtering methods may also be used. The filtering method used is not limited as long as the purpose of optimizing the second followed set may be achieved.", + "p0175": "Then the fourth followed set is matched with each of the third followed sets. When the matching degree of the fourth followed set corresponding to the one or more newly registered users and the third followed set is greater than the preset threshold or the third followed set is exactly the same, it is determined that the newly registered user is matched with the third followed set so as to determine the registered user set to which the newly registered user belongs.", + "p0176": "The recommendation label recommended for the new user is determined according to the user set label dictionary of the registered user set to which the newly registered user belongs.", + "p0177": "In the practical applications, after mining a group interest model of a user group having similar interest to that of the newly registered users, we can integrate the group interest model and the user individual interest model according to a certain weight, and then recommend the content according to the integrated interest model. Specifically, given an integrated interest model , we can recommend some of the highest quality content under the label according to the ratio such as the weight of each interest label.", + "p0178": "It is to be noted that, for a new user, we cannot acquire the individual interest model of the new user without any of the act data of the user in the station. But if the new user logs in the information client with a network account of the social platform, we can acquire the social relations of the newly registered user on the social platform. The targeted recommended information may be achieved by mining the user group having similar interest within the station and using this group interest model to recommend content to users. In practice, this practice has a better effect than randomly recommending or recommending the most popular content.", + "p0179": "Preferably, in the preferred embodiment provided by the disclosure, as shown in FIG. 6 , the device further includes a pushing component 43 .", + "p0180": "Herein the pushing component 43 is arranged to push the recommended information for the one or more newly registered users according to the one or more recommendation interest labels.", + "p0181": "Specifically, through the pushing component 43 , the recommended information matching the interest label is pushed for the newly registered user according to the interest label determined for the newly registered user through the above steps.", + "p0182": "As may be seen from the technical solution, the embodiments of the disclosure effectively combine the SNS public data and the recommended service private data together as the user recommended content. Compared with using only the SNS public data or the recommended service private data, the integration of the two data is conducive to more accurately recommending personalized content. Moreover, the integrating method proposed by the embodiments of the disclosure can also utilize the integration of the two data , which is also the effect that the traditional method cannot achieve.", + "p0183": "One feature of the embodiments of the disclosure is that if the recommended service provider has more users, the effect of this method is the better. Because the user group of such a service provider has a relatively large coverage for user groups of the SNS. The situation will not occur that a use randomly gives a social account and most of his/her friends or fans are not users in the station so that the group interest cannot be mined. This is a significant competitive advantage for products such as headlines today that have 100 million users, but it is a technical barrier for some smaller recommended products.", + "p0184": "Each of the functional units provided in the embodiment of the disclosure may be operated in a mobile terminal, a computer terminal or a similar arithmetic device, or may be stored as a part of the storage medium.", + "p0185": "Thus, embodiments of the disclosure may provide a computer terminal that may be any of computer terminal devices in a computer terminal group. Optionally, in the present embodiment, the above computer terminal may be replaced with a terminal device such as a mobile terminal.", + "p0186": "Optionally, in the present embodiment, the above computer terminal may be located in at least one of the plurality of network devices in the computer network.", + "p0187": "In the present embodiment, the above computer terminal may execute the program code of the following steps in the method for social platform-based data mining: acquiring one or more interest label dictionaries of one or more registered users on an information client; acquiring one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and reading the relationship information between the one or more registered users and the one or more first objects; according to the one or more first objects having the followed relationship with the one or more registered users, determining one or more first followed sets corresponding to the one or more registered users; according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, constructing an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; acquiring one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and reading relationship information between the one or more newly registered users and the one or more second objects; according to the one or more second objects having the followed relationship with the one or more newly registered user, determining a second followed set corresponding to the one or more newly registered users; and matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0188": "Optionally, the computer terminal may include one or more processors, memories, and transmitting devices.", + "p0189": "Herein the memory may be used to store software programs and components, such as program instructions/components corresponding to the method for social platform-based data mining in the embodiments of the disclosure, and the processor performs various function applications and data processing by running software programs and components stored in the memory, that is, achieving the above method for social platform-based data mining. The memory may include a high-speed random access memory, and may include a non-volatile memory such as one or more magnetic storage devices, flash memories, or other non-volatile solid state memories. In some examples, the memory may further include a memory remotely provided with respect to the processor, which may be connected to the terminal via a network. Examples of the above networks include, but are not limited to, the Internet, the intranet, the local area network, the mobile communication network, and combinations thereof.", + "p0190": "The above transmitting device is for receiving or transmitting data via a network. The specific example of the above network may include a wired network and a wireless network. In one example, the transmitting device includes a Network Interface Controller that may be connected to the router via a network cable and other network devices to communicate with the Internet or the local area network. In one example, the transmitting device is a Radio Frequency component for wirelessly communicating with the Internet.", + "p0191": "Herein specifically, the memory is used for storing information of a preset action condition and a preset privileged user and an application program. The processor may call the information and application program stored in the memory by means of the transmitting device to execute the program code of the method steps of various optional or preferred embodiments in the above method embodiments.", + "p0192": "It will be understood by those of ordinary skill in the art that computer terminals may also be terminal devices such as smart phones , tablet PCs, palm computers, Mobile Internet Devices , and PADs.", + "p0193": "It will be understood by those of ordinary skill in the art that all or a part of steps in the various methods of the above embodiments may be accomplished by a program instructing the terminal device related hardware, and the program may be stored in a computer readable storage medium. The storage medium may include: a flash disk, a Read-Only Memory , a Random Access Memory , a magnetic disk or an optical disk.", + "p0194": "The embodiment of the disclosure also provides a storage medium. Alternatively, in the embodiment, the above storage medium may be used for storing the program code executed by the method for social platform-based data mining provided by the method embodiment described above.", + "p0195": "Optionally, in the embodiment, the above storage medium may be located in any of the computer terminals in the computer terminal group in the computer network or in any of the mobile terminals in the mobile terminal group.", + "p0196": "Optionally, in the embodiment, the storage medium is arranged to store the program code for performing the following steps: acquiring one or more interest label dictionaries of one or more registered users on an information client; acquiring one or more first objects having followed relationship with the one or more registered users on the information client in a social platform and reading the relationship information between the one or more registered users and the one or more first objects; according to the one or more first objects having the followed relationship with the one or more registered users, determining one or more first followed sets corresponding to the one or more registered users; according to the one or more interest label dictionaries of the one or more registered users and the one or more first followed sets, constructing an interest model, wherein the interest model is used to characterize a corresponding relationship between the one or more registered users having the same first followed set and an interest label; acquiring one or more second objects having followed relationship with one or more newly registered users on the information client in the social platform, and reading relationship information between the one or more newly registered users and the one or more second objects; according to the one or more second objects having the followed relationship with the one or more newly registered user, determining a second followed set corresponding to the one or more newly registered users; and matching the second followed set with the interest model to determine one or more recommended interest labels of the one or more newly registered users according to the interest model.", + "p0197": "Optionally, in the present embodiment, the storage medium may also be arranged to store the program code of various preferred or optional method steps provided by the method for social platform-based data mining.", + "p0198": "The method for social platform-based data mining according to the embodiments of the disclosure is described in an exemplary manner with reference to the accompanying drawings. However, it will be understood by those skilled in the art that various modifications may be made to the above page layout method and system proposed in the embodiments of the disclosure without departing from the content of the embodiments of the disclosure. Accordingly, the scope of protection of the embodiments of the disclosure should be determined by the content of the appended claims.", + "p0199": "The above is only the preferred embodiment of the disclosure and not intended to limit the embodiments of the disclosure, and for the technician of the field, the embodiments of the disclosure may have various modifications and variations. Any modifications, equivalent replacements, improvements and the like within the spirit and principle of the embodiments of the disclosure shall fall within the scope of protection as defined in the embodiments of the disclosure.", + "features": { + "F1": "A method for social platform-based data mining comprising acquiring one or more interest label dictionaries of registered users on an information client with a computer terminal.", + "F2": "Acquiring, by the computer terminal, first objects having a followed relationship with registered users on the information client in a social platform and reading relationship information between registered users and first objects.", + "F3": "Determining, by the computer terminal, first followed sets corresponding to registered users based on first objects having a followed relationship with registered users.", + "F4": "Constructing, by the computer terminal, an interest model using registered users' interest label dictionaries and first followed sets to characterize a relationship between registered users sharing the same followed set and an interest label.", + "F5": "Acquiring, by the computer terminal, second objects related to newly registered users on the information client in a social platform, and reading relationship information between newly registered users and second objects.", + "F6": "Determining, by the computer terminal, a second followed set for newly registered users based on second objects having a followed relationship with newly registered users.", + "F7": "Matching, by the computer terminal, the second followed set with the interest model to determine recommended interest labels for newly registered users according to the interest model.", + "F8": "Pushing recommended information to newly registered users based on recommended interest labels.", + "F9": "Acquiring recommended information and extracting interest labels from its content before acquiring interest label dictionaries of registered users.", + "F10": "Acquiring historical behavior data of registered users to record operational behavior for recommended information.", + "F11": "Determining label weight values of interest labels according to historical behavior data.", + "F12": "Determining interest label dictionaries for registered users based on label weight values.", + "F13": "Performing filtering in the first followed set to acquire a third followed set using data, index, condition, or information filtering methods.", + "F14": "Matching registered users based on the third followed set to generate a registered user set comprising registered users with the same third followed set.", + "F15": "Generating a user set label dictionary for the registered user set using interest label dictionaries of registered users in the set.", + "F16": "Acquiring a first user amount on the information client and a second user amount in the registered user set.", + "F17": "Calculating a weight distribution average value for interest labels using label weight values and the first user amount.", + "F18": "Calculating a set weight average value for interest labels using label weight values and the second user amount of registered users in the set.", + "F19": "Calculating a registered user set weight value for interest labels using the weight distribution average value and the set weight average value.", + "F20": "Comparing registered user set weight values of interest labels with a preset noise threshold to retain or delete interest labels in the user set label dictionary.", + "F21": "Performing filtering in the second followed set to acquire a fourth followed set using data, index, condition, or information filtering methods.", + "F22": "Matching the fourth followed set with the third followed set to determine the registered user set for newly registered users.", + "F23": "Determining recommended interest labels for newly registered users using the user set label dictionary of their corresponding registered user set.", + "F24": "Determining registered user set weight value V\u2032 of interest labels using the formula V\u2032 = V / V base, where V\u2032 is the registered user set weight value, V is the set weight average value, and V base is the weight distribution average value.", + "F25": "A computer terminal configured to execute program code of steps for social platform-based data mining as claimed in the method.", + "F26": "A storage medium for storing program code executed by the method for social platform-based data mining as claimed in the method.", + "F27": "A device for social platform-based data mining comprising a hardware processor configured to execute program units stored on memory.", + "F28": "The device includes a first acquiring component to acquire interest label dictionaries of registered users on an information client.", + "F29": "The device includes a second acquiring component to acquire first objects with followed relationships and read relationship information between users and objects.", + "F30": "The device includes a first determining component to determine first followed sets for registered users based on followed relationships.", + "F31": "The device includes a first processing component to construct an interest model characterizing user relationships using interest label dictionaries and first followed sets.", + "F32": "The device includes a third acquiring component to acquire second objects for newly registered users and read relationship information.", + "F33": "The device includes a second determining component to determine second followed sets for newly registered users.", + "F34": "The device includes a second processing component to match second followed sets with the interest model to determine recommended interest labels.", + "F35": "The device is configured to push recommended information to newly registered users based on recommended interest labels.", + "F36": "The device includes a fourth acquiring component to acquire recommended information.", + "F37": "The device includes an extracting component to extract interest labels from recommended information content.", + "F38": "The device includes a fifth acquiring component to acquire historical behavior data of registered users.", + "F39": "The device includes a third determining component to determine label weight values using historical behavior data.", + "F40": "The device includes a fourth determining component to determine interest label dictionaries using label weight values.", + "F41": "The device includes a first processing sub-component to perform filtering in the first followed set to acquire a third followed set.", + "F42": "The device includes a matching sub-component to match registered users based on the third followed set to generate a registered user set.", + "F43": "The device includes a first generating component to generate a user set label dictionary for the registered user set.", + "F44": "The device includes a first acquiring sub-component to acquire user amounts for individuals on an information client and in a registered user set.", + "F45": "The device includes first, second, and third calculating sub-components for computing weight distribution average values, set weight average values, and registered user set weight values, respectively.", + "F46": "The device includes a judging sub-component for comparing registered user set weight values with a preset noise threshold.", + "F47": "The device includes a second processing sub-component to perform filtering in the second followed set to acquire a fourth followed set.", + "F48": "The device includes a first determining sub-component to match fourth followed sets with third followed sets to determine registered user sets.", + "F49": "The device includes a second determining sub-component to determine recommended interest labels for newly registered users using user set label dictionaries.", + "F50": "The device includes a pushing component to push recommended information to newly registered users based on recommendation interest labels." + } + } + }, + { + "FAN": "70563808", + "CPC": [ + "G06F-009/44/28", + "G06F-009/4488", + "G06F-009/455/29", + "G06F-016/00", + "G06F-017/30", + "G06F-021/62/27", + "H04L-009/08", + "H04L-009/08/25", + "H04L-009/32/36", + "H04L-009/32/47", + "H04L-009/32/63", + "H04L-063/0428", + "H04L-063/0442", + "H04L-063/061", + "H04L-063/0823", + "H04L-063/123", + "H04L-2209/30", + "H04L-2463/061", + "H04L-2463/062" + ], + "Content": { + "title": "Extended javascript object notation file generator circuitry and process", + "pa01": "Methods and apparatus to provide extended object notation data are disclosed. An example apparatus includes a data handler having a first input to receive object data and a first output to output an object notation key-value pair for the object data; a string processor having a second input coupled to the first output and a second output to convey the object notation key-value pair without string literals; and a hashing and encryption handler having a third input coupled to the second output and a third output to convey the key-value pair signed with a private key, to convey the key-value pair encrypted with a public key, and to convey an indication that the encrypted key-value pair is encrypted in a key of the encrypted key-value pair.", + "c-en-0001": "A process of producing an extended JavaScript Notation file comprising: determining a cipher and a key to be used in at least one of hashing and encrypting a JavaScript Object Notation key-value pair; signing the key-value pair using the determined key; encrypting the signed key-value pair when the cipher includes encryption; converting the encrypted key-value pair to a string for insertion in an extended JavaScript Object Notation file; inserting an encryption identifier in the key of the key-value pair; inserting metadata identifying the cipher in the xJSON file; determining a hash for the value of the key-value pair and hashing the key-value pair; and inserting the value of the hash into the key for the key-value.", + "c-en-0002": "The process of claim 1 in which determining a cipher includes determining whether the cipher includes encryption, hashing, or hashing and encryption.", + "c-en-0003": "The process of claim 1 in which determining a cipher includes determining whether the cipher includes encryption, hashing or hashing and encryption and when the cipher does not include encryption, proceeding to determining a hash for the value of the key-value pair.", + "c-en-0004": "The process of claim 1 in which the converting includes transferring the encrypted value to a string using Base64 encoding.", + "c-en-0005": "The process of claim 1 in which the inserting an encryption identifier in the key of the key-value pair includes inserting an identifier \u201c \u201d in the key.", + "c-en-0006": "The process of claim 1 in which the inserting metadata includes inserting the metadata in a key-value pair with key name \u201ckeys\u201d.", + "c-en-0007": "The process of claim 1 including determining whether there are multiple ciphers in the keys metadata.", + "c-en-0008": "The process of claim 1 including determining whether there are multiple ciphers in the keys metadata and if there are multiple ciphers in the keys metadata, inserting a cipher identifier in the key of the encrypted key-value pair.", + "c-en-0009": "The process of claim 1 in which inserting metadata includes inserting an index corresponding to the cipher in the keys metadata.", + "c-en-0010": "The process of claim 1 in which inserting metadata includes inserting an index \u201c \u201d corresponding to the cipher in the keys metadata, where the cipher is the second cipher in the keys metadata.", + "c-en-0011": "The process of claim 1 in which determining a hash for the value of the key-value pair and hashing the key-value pair occurs after determining that the cipher does not include encryption, after determining that there are not multiple ciphers, or after inserting the identifier of the cipher in the key.", + "c-en-0012": "The process of claim 1 including receiving a request to hash and/or encrypt that includes an identification of a cipher, a key, or a private key.", + "c-en-0013": "Extended JavaScript Object Notation generator circuitry comprising: data handler circuitry having a first input receiving object data, a second input, and a file output, the data handler circuitry records the content type for an extended JavaScript Object Notation file, creates a key-value pair for the object data, by creating a key named for the variable of the object data and creating a corresponding value for the value of the variable, and sends and the key-value pair to the output; string processor circuitry having an input coupled to the file output and an output, the string processor circuitry processing the object notation data to be parsed by xJSON capable devices; and hashing and encryption handler circuitry having an input coupled to the output of the string processor circuitry and an output coupled to the second input of the data handler circuitry, the hashing and encryption handler circuitry: determining a cipher and a key to be used in at least one of hashing and encrypting a JavaScript Object Notation key-value pair; signing the key-value pair using the determined key; encrypting the signed key-value pair when the cipher includes encryption; converting the encrypted key-value pair to a string for insertion in an extended JavaScript Object Notation file; inserting an encryption identifier in the key of the key-value pair; inserting metadata identifying the cipher in the xJSON file; determining a hash for the value of the key-value pair and hashing the key-value pair; and inserting the value of the hash into the key for the key-value.", + "c-en-0014": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry determines whether the cipher includes encryption, hashing, or hashing and encryption.", + "c-en-0015": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry determines whether the cipher includes encryption, hashing or hashing and encryption and when the cipher does not include encryption, proceeding to determining a hash for the value of the key-value pair.", + "c-en-0016": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry transfers the encrypted value to a string using Base64 encoding.", + "c-en-0017": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry inserts an encryption identifier in the key of the key-value pair by inserting an identifier \u201c \u201d in the key.", + "c-en-0018": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry inserts the metadata in a key-value pair with key name \u201ckeys\u201d.", + "c-en-0019": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry determines whether there are multiple ciphers in the keys metadata.", + "c-en-0020": "The extended JavaScript Object Notation generator circuitry of claim 13 in which the hashing and encryption handler circuitry determines whether there are multiple ciphers in the keys metadata and if there are multiple ciphers in the keys metadata, inserts a cipher identifier in the key of the encrypted key-value pair.", + "p0001": "RELATED APPLICATION", + "p0002": "This application is a divisional of prior application Ser. No. 14/611,080, filed Jan. 30, 2015, now U.S. Pat. No. 9,853,956, issued Dec. 26, 2017.", + "p0003": "This patent claims priority to U.S. Provisional Patent Application Ser. No. 61/938,381, filed Feb. 11, 2014. The entirety of U.S. Provisional Patent Application Ser. No. 61/938,381 is hereby incorporated herein by reference.", + "p0004": "FIELD OF THE DISCLOSURE", + "p0005": "This disclosure relates generally to object notation data and, more particularly, to methods and apparatus to provide extended object notation data.", + "p0006": "BACKGROUND", + "p0007": "In network communications it is often beneficial to utilize a communication standard that utilizes human-readable text. Such communication standards are often easier for programmers to understand and may be more flexible than application specific binary formats. One example communication standard that utilizes human-readable text is JavaScript Object Notation . JSON is well-suited for Internet communications because of its close ties to JavaScript, which is supported out-of-the-box by many Internet browsers and other applications.", + "p0008": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0009": "FIG. 1 is a block diagram of an example environment in which example methods and apparatus disclosed herein may be implemented to generate and/or parse xJSON and/or any other human-readable object notation data files.", + "p0010": "FIG. 2 is a block diagram of an example implementation of the example xJSON handler of FIG. 1 .", + "p0011": "FIG. 3 is a block diagram of an example implementation of the example generator of FIG. 2 .", + "p0012": "FIG. 4 is a block diagram of an example implementation of the example parser of FIG. 2 .", + "p0013": "FIGS. 5-8 are flowcharts representative of an example computer readable instructions that may be performed to generate extended JSON data.", + "p0014": "FIGS. 9-11 are flowcharts representative of an example computer readable instructions that which may be performed to parse extended JSON data.", + "p0015": "FIG. 12 is a block diagram of an example processor platform structured to execute the instructions of FIGS. 5-11 to implement the generator and/or the example parser of FIGS. 2-4 .", + "p0016": "DETAILED DESCRIPTION", + "p0017": "The figures are not to scale. Wherever appropriate, the same reference numbers will be used throughout the drawing and accompanying written description to refer to the same or like parts.", + "p0018": "DETAILED DESCRIPTION", + "p0019": "The Internet of Things refers to the concept of joining a wide range of devices to the Internet. The \u201cthings\u201d may be any type of device or system, which often includes many devices that have previously not included circuitry capable of communicating on a network such as the Internet . For example, a coffee maker may include an embedded computing device that allows the coffee maker to be uniquely identified on the Internet and allows remote control and monitoring of the example coffee maker via other Internet connected devices. Many IoT devices include low-cost and/or low-power computing devices to reduce the cost and physical space needed to add IoT functionality.", + "p0020": "While JSON and other standards utilizing human-readable text for storing and transmitting data objects , Yet Another Markup Language , etc.) are well-suited for use with devices communicating on the Internet, example methods and apparatus disclosed in this application provide extensions to such human-readable formats to facilitate the use of the human-readable protocols with limited-resource devices such as IoT devices. This is advantageous because the disclosed methods and apparatus facilitate use of the desirable object notation data formats with IoT devices and/or any other device that has limited computing resources and/or communicates with many diverse devices.", + "p0021": "While the extensions disclosed herein are well-suited for use with IoT devices, the extensions are not limited to use with and/or by IoT devices. Examples disclosed herein are described with reference to an extended JSON, which is referred to herein as xJSON for consistency. Alternatively, the extended JSON may be utilized with any other content type name and/or the extensions may be utilized with an extended version of any other protocol or standard. The methods and apparatus disclosed herein at not limited to extending JSON. Rather, the extensions may be utilized with any type of human-readable based protocol for storing and transmitting objects.", + "p0022": "In JSON, objects are denoted by an array of key-value pairs delimited with opening and closing curly brackets. A key denotes a property of the object and the value identifies the value for that property. Keys and values are separated by a colon. For example, a person object in JSON may be stored in a file as:", + "p0024": "In the above example, firstName, lastName, email, and password are keys and John, Smith, john.smith@example.com, and secretPassword123 are values. The keys in the object may be referred to as names and may correlate with variables that store the values when the object is stored in an application . Thus, the JSON object provides a way to represent an object stored in binary or any other format in a human readable format.", + "p0025": "The example extensions described herein include data packing, object serialization, and hashing/security.", + "p0026": "As used herein, data packing refers to applying a compression algorithm to keys and/or values ). When a xJSON file is received, packed data in the xJSON file may be identified and decompressed.", + "p0027": "As used herein, object serialization refers to converting keys and/or values to a binary value . In some examples, the binary value are then converted to a text format . When a xJSON file is received, serialized data in the xJSON file may be detected and deserialized/unmarshalled.", + "p0028": "In some examples, hashing/security operations are performed by generating a hash for a value of a key-value pair and inserting the hash into the key. The hash can be used for validating the contents of the value by comparing the hash stored in the key with a hash generated for the value of the key-value pair. The hash may additionally or alternatively be used for searching for data contained in values. For example, a search parameter can be hashed and the hash for the search parameter can be compared with the hashes stored in keys to identify a match and, therefore, a value of a key-value pair that matches the search parameter. Additionally or alternatively, hashing/security may also include encrypting keys and/or values and replacing the unencrypted keys and/or values with the encrypted data. When a xJSON file is received, encrypted data in the xJSON file may be detected and decrypted.", + "p0029": "In example methods and apparatus disclosed herein, in applying the disclosed extensions to JSON, xJSON capable devices insert a qualifier in a key and/or value when the key and/or the value have been generated and/or modified based on xJSON extensions. The qualifier indicates to other xJSON capable devices that the xJSON extension has been applied. For example, a key may be modified by adding brackets to the end of the key and inserting a qualifier in the brackets. For example, if an extension associated with the character \u201cx\u201d is applied to key-value pairs in the person object shown in the previous paragraph, the xJSON representation may be:", + "p0031": "The qualifier may alternatively be inserted between brackets added to the value of the key-value pair and/or qualifier may be added to both the key and the value . Alternatively, delimiters other than brackets may be utilized to separate the identifier from the key and/or value .", + "p0032": "For consistency the example disclosed herein are described with reference to xJSON identifiers inserted between brackets in the key of key-value pairs. However, this disclosure is not limited to a particular format for the insertion of the identifiers and any other format, including those described above, may be utilized.", + "p0033": "The insertion of the identifier in the key name and/or value ensures that the xJSON representation can still be processed by a device that supports JSON but does not support xJSON . Accordingly, the use of xJSON will not cause devices that support JSON, but not xJSON, to fail during parsing of an xJSON file. Rather, these non-xJSON devices will continue to operate, but without an understanding of the extensions. Such an approach enhances the ability for xJSON capable devices to operate in an environment in which some devices do not support xJSON.", + "p0034": "An additional advantage of inserting the identifier in the key as disclosed herein allows xJSON extensions to be applied on a selective basis. For example, an xJSON extension may be applied to an entire file, may be selectively applied to one or more objects in a file, may be selectively applied to one or more keys-value pairs in a file, etc. Thus, when the xJSON file is being processed, key-value pairs that include an xJSON identifier in the key can be processed according to the extension and key-value pairs that do not include an xJSON identifier in the key can be processed using standard JSON processing. Furthermore, different ones of the extensions can be applied to subsets of the key-value pairs in a file. For example, in the foregoing example \u201cperson\u201d object, the firstName, lastName, and email key-value pairs may be processed to insert a hash value and an xJSON identifier for hashing in the corresponding keys, the password key-value pair may be encrypted and hashed, and an encryption identifier, a hash identifier, and a hash value may be inserted in the corresponding key.", + "p0035": "FIG. 1 is a block diagram of an example environment 100 in which example methods and apparatus disclosed herein may be implemented to generate and/or parse xJSON and/or any other human-readable object notation data files. The example environment includes an example web service 102 to convey an example xJSON data 103 via an example network 104 to an example first device 106 and an example second device 108 . As used herein, the phrase \u201cin communication,\u201d including variances thereof, encompasses direct communication and/or indirect communication through one or more intermediary components and does not require direct physical communication and/or constant communication, but rather additionally includes selective communication at periodic or aperiodic intervals, as well as one-time events.", + "p0036": "In the illustrated example, the example web service 102 , the example first device 106 , and the example second device 108 exchange data using JSON data files. According to the illustrated example, the example web service 102 and the example first device 106 are xJSON capable in that they include an xJSON handler 110 to parse and/or generate files that are based on at least one of the extensions associated with xJSON disclosed herein. According to the illustrated example, the example second device 108 is not xJSON capable in that the example second device 108 may parse and/or generate JSON files but does not include the xJSON handler 110 for parsing and/or generating xJSON files with the extensions associated with xJSON disclosed herein. As disclosed herein, while the example second device 108 is not capable of utilizing the extensions related to xJSON, the example xJSON data 103 of the illustrated example that is output by the example web service 102 and/or the example first device 106 may be successfully parsed .", + "p0037": "In the illustrated example, the web service 102 is a server computer for serving information on the Internet. Alternatively, the web service 102 may be any type of device with which a device connected to the example network 104 may communicate. The example web service 102 sends the example xJSON data 103 to the example first device 106 and/or the example second device 108 . The example web service 102 may also receive xJSON data from the example first device 106 and/or JSON data from the example second device 108 . Alternatively, the example web service 102 may only be capable of receiving data or may only be able of sending data .", + "p0038": "The network 104 of the illustrated example of FIG. 1 is the Internet. Alternatively, the network 104 may be any type of and/or combination of a local area network, a wide area network, a wired network, a wireless network, a private network, a public network, etc. The example network 104 communicatively couples the example web service 102 with the example first device 106 and the example second device 108 .", + "p0039": "The first device 106 of the illustrated example of FIG. 1 is an IoT device that includes the example xJSON handler 110 to parse and/or generate the example xJSON data 103 . For example, the first device 106 may be a network-enabled microprocessor controller. For example, the first device 106 may be the CC3100 SimpleLink\u2122 Wi-Fi\u00ae and Internet-of-Things Solution for MCU Applications or the CC3200 SimpleLink\u2122 Wi-Fi\u00ae and Internet-of-Things Solution, a Single-Chip Wireless MCU from Texas Instruments\u00ae and/or a device that includes the CC3100 or the CC3200. Alternatively, the first device 106 may be any other device in which it is desirable to utilize JSON data.", + "p0040": "The second device 108 of the illustrated example of FIG. 1 may be any device in which it is desirable to utilize JSON data. The second device 108 is included in the example of FIG. 1 to illustrate that devices that support xJSON extensions and devices that do not support xJSON extensions may be connected to the same network and may communicate with each other. For example, when xJSON extensions are implemented in a manner that does not run afoul of the JSON grammar, JSON files that include at least some key-value pairs that include xJSON extensions can be parsed by devices that do not support xJSON without causing parsing errors. Likewise, devices that support xJSON extensions are able to process JSON files.", + "p0041": "The example xJSON handler 110 parses and/or generates xJSON files . An example implementation of the xJSON handler 110 is described in further detail in conjunction with FIG. 2 . While FIG. 1 illustrates that the example web service 102 and the example first device 106 include the same xJSON handler 110 , in other examples, devices may include different xJSON handlers .", + "p0042": "Utilizing the xJSON handler 110 enables a device to generate xJSON data and parse xJSON data . The xJSON handler 110 of the illustrated example facilitates the use of data representations that are not supported by existing object notation protocols. For example, the xJSON handler 110 may support the use of customized primitives . For example, a binary typed literal may be input as \u201c0bAAAA\u201d or \u201c0BAAAA\u201d where \u201cb\u201d and \u201cB\u201d indicate that the value is a binary literal. In another example, a hexadecimal typed literal may be input as \u201c0xAAAA\u201d or 0XAAAA\u201d where \u201cx\u201d and \u201cX\u201d indicate that the value is a hexadecimal literal. Hardware based literals may also be supported by the xJSON handler 110 . For example, an identifier may be added to a key and/or a value to indicate a literal of a volatile type, a literal for a hardware signal type , and a tri-state value for a signal object. In other words, the flexibility of utilizing identifiers appended to, inserted in, replacing portions of keys and/or values, etc. allows the xJSON handler 110 to indicate information about keys and/or values including indicating the state and/or the purpose of the value .", + "p0043": "While the example environment 100 of FIG. 1 includes the example web service 102 , the example first device 106 , and the example second device 108 , any number and/or types of devices may be utilized. For example, an environment might include any combination of two devices and/or web services, three devices and/or web services, four devices and/or web services, hundreds of devices and/or web services, etc.", + "p0044": "FIG. 2 is a block diagram of an example implementation of the example xJSON handler 110 of FIG. 1 . The example xJSON handler 110 of FIG. 2 includes an example interface 202 to send and/or receive example object notation data 200 , an example parser 204 to parse the example object notation data 200 to output example object data 210 , an example generator 206 to generate example object notation data 200 from example object data 212 , and an example JavaScript Interpreter 208 . While the xJSON handler 110 of FIG. 2 is described with reference to the example first device 106 , the xJSON handler 110 of FIG. 2 may be implemented in another device .", + "p0045": "The example interface 202 of the illustrated example is a network interface 202 that sends and/or receives the object notation data 200 to and/or from the network 104 and/or from one or more other components of the device that includes a xJSON handler 110 . For example, the xJSON handler 110 of the example first device 106 may receive the example xJSON data 103 retrieved from the example web service 102 . The example interface 202 transmits the object notation data 200 received from the network 104 to the example parser 204 . The example interface 202 transmits the object notation data 200 received from the example generator 206 to a desired destination for the object notation data 200 . For example, the interface 202 for the xJSON handler 110 of the example first device 106 may transmit the example object notation data 200 generated by the example generator 206 to the example web service 102 .", + "p0046": "The parser 204 of the illustrated example receives the object notation data 200 and parses the data to extract the objects represented by the object notation data 200 . The parser 204 transmits extracted object data 210 to the example JavaScript interpreter 208 . For example, returning to the example person object discussed above, the parser 204 retrieves the elements of the person object from the key-value pairs included in the xJSON data and builds a JavaScript person object that is transmitted to the JavaScript interpreter 208 . The example parser 204 includes functionality for parsing xJSON data that includes one or more of data packing, object serialization, and/or hashing/security extensions. An example implementation of the parser 204 is described in further detail in conjunction with the block diagram of FIG. 4 .", + "p0047": "The example generator 206 of the illustrated example builds object notation data 200 file to represent object data 212 received from the JavaScript interpreter 208 . For example, the example generator 206 may build the example person object in xJSON based on a person object stored in the JavaScript interpreter. The object notation data 200 generated by the generator 206 is transmitted to the interface 202 for transmission to another device . Alternatively, the xJSON handler 110 and/or a device that includes the xJSON handler 110 may store the object notation data 200 . The example generator 206 is described in further detail in conjunction with the block diagram of FIG. 3 .", + "p0048": "The JavaScript interpreter 208 of the illustrated example is a software run-time environment that operates according to the JavaScript programming language to execute JavaScript applications or any other JavaScript instructions. The example JavaScript interpreter 208 of the illustrated example stores object data 210 and/or 212 . While the JavaScript interpreter 208 of the illustrated example utilizes JavaScript, the JavaScript interpreter 208 may alternatively be any other run-time environment that can receive objects output by the example parser 204 and/or transmit objects to the example generator 206 .", + "p0049": "FIG. 3 is a block diagram of an example implementation of the example generator 206 of FIG. 2 . The example generator 206 of FIG. 3 includes an example data handler 302 , an example string processor 304 , an example hashing and encryption handler 306 , an example compression handler 308 , and an example serialization processor 310 .", + "p0050": "The example data handler 302 receives object data 312 and generates an object notation data 314 populated with xJSON key-value pairs representative of the object data 312 . For example, the example data handler 302 may provide an interface ) through which a request for generation of an xJSON file may be received. The example data handler 302 determines if the request indicates that an xJSON file is to be generated or if a JSON file that includes xJSON extensions is to be generated. For example, as described in detail herein, if the object notation data 314 does not need to be compatible with devices that do not support xJSON, the object notation data 301 output by the generator 206 may be formatted to be processed by an xJSON capable device . The example data handler 302 of the illustrated example records the content type for the xJSON file, creates a key-value pair 316 for the object data 312 , and sends and the key-value pair 316 to the example string processor 304 .", + "p0051": "The example string processor 304 of FIG. 3 determines if the content type for the object notation data 314 is to be xJSON or standard JSON. If the object notation data 314 is intended to be parsed by xJSON capable devices and non-xJSON capable devices, the string processor 304 inserts quotation marks around the keys and the values in the key-value pair 316 . If compatibility with non-xJSON capable devices was not desired, the string processor 304 does not insert the quotation marks as the strings of the example key-value pair 316 will be implicitly recognized by xJSON capable parsers . For example, the following example person object may be generated when non-xJSON compatibility is desired:", + "p0052": ".", + "p0053": "Alternatively, the following example person object may be generated when non-xJSON compatibility is not desired and/or needed:", + "p0054": ".", + "p0055": "The example string processor 304 outputs a processed key-value pair 318 to the hashing and example encryption handler 306 .", + "p0056": "The example hashing and encryption handler 306 of FIG. 3 receives the processed key-value pair 318 and determines if hashing and/or encryption of the processed key-value pair 318 is requested. For example, the request to generate the object notation data 314 may identify one or more key-value pairs and/or objects for which hashing and/or encryption is requested. For example, if an object includes a username field and a password field, hashing may be requested for all fields but encryption may be requested for only the password field. Alternatively, the hashing and encryption handler 306 may automatically determine that hashing and/or encryption is desired .", + "p0057": "When hashing and/or encryption are requested, the hashing and encryption handler 306 determines a desired cipher to be used. For example, the request to perform hashing and/or encryption may identify a cipher and/or the example hashing and encryption handler 306 may utilize a default cipher.", + "p0058": "To certify the authenticity of the processed key-value pair 318 to other devices, the hashing and encryption handler 306 of this example signs the processed key-value pair 318 utilizing a private key of the content-owner to generate an encrypted and/or hashed key-value pair 320 . In such examples, the encrypted and/or hashed key-value pair 320 can be verified by others with access to the public key corresponding to the private key.", + "p0059": "When the cipher includes encryption , the example hashing and encryption handler 306 of FIG. 3 encrypts the encrypted and/or hashed key-value pair 320 for any key-value pairs for which encryption was requested. The example hashing and encryption handler 306 encrypts the encrypted and/or hashed key-value pair 320 using a public key corresponding to a private key that may be utilized to decrypt the encrypted and/or hashed key-value pair 320 . For example, the parser 204 of FIGS. 2 and/or 4 may store a private key that may be utilized for decrypting data encrypted using a corresponding public key.", + "p0060": "The example hashing and encryption handler 306 then hashes the encrypted and/or hashed key-value pair 320 . The example hashing and encryption handler 306 hashes the encrypted value for any encrypted data .", + "p0061": "The hashing and encryption handler 306 of the illustrated example inserts cipher keys data into the encrypted and/or hashed key-value pair 320 that is being generated. The cipher keys data identifies one or more keys that were utilized in encrypting the encrypted and/or hashed key-value pair 320 . For example, the cipher keys data may include an identifier for a certificate for which a public key was used to encrypt the encrypted and/or hashed key-value pair 320 to enable the example parser 204 to locate the corresponding private key for use in decrypting the encrypted and/or hashed key-value pair 320 . The cipher keys may additionally include an identifier for a certificate for which a private key was utilized for signing the data. Where multiple keys are utilized in a single xJSON file, each key may be identified with a sequential number in the cipher keys data. In addition to the identifier for the encryption key, the cipher keys data of the illustrated example also identifies the particular cipher algorithm used for the hashing and/or encryption and any parameters corresponding to the cipher algorithm.", + "p0062": "In some examples, the hashing and encryption handler 306 also inserts the public key certificate that were utilized for encrypting the encrypted and/or hashed key-value pair 320 and/or that may be utilized to validate the signing data in the encrypted and/or hashed key-value pair 320 .", + "p0063": "The example hashing and encryption handler 306 of the example of FIG. 3 inserts in the corresponding keys of the encrypted and/or hashed key-value pair 320 an indication of the hashing and/or an indication of the encryption. For example, for a key that has its value hashed, the example hashing and encryption handler 306 inserts \u2018 \u2019 in the key, where \u2018#\u2019 is a hash identifier and \u2018HHHH\u2019 is the value resulting from the hashing. The example hashing and encryption handler 306 inserts \u2018 \u2019 in the key for the encrypted and/or hashed key-value pair 320 , where the \u2018s\u2019 is a hash identifier, the \u2018X\u2019 is an index value of the key utilized for the signing and/or encryption where there are multiple keys, the \u2018#\u2019 is a hash identifier, and \u2018HHHH\u2019 is the value resulting from the hashing.", + "p0064": "For example, the following is a person object before hashing and encryption:", + "p0066": "A hashed and encrypted person object that may be generated by the example hashing and encryption handler 306 of FIG. 3 for the person object above is:", + "p0067": "}]; \u2033cert\u2033: \u2033name \u2033: \u2033John Smith\u2033, \u2033email \u201d: \u201cjohn.smith@example.com\u2033, \u2033passwords \u2033:\u2033MzA0NTAyMjEwMGQzNmYzY2EyZjJmNjc4YTRmMGEyNWVjO DdhMDk1MjA4YTJhMjViZWRlZTRiYzFhZDdkNzIwNzU1NzMzMjYyNGEwMjIwNGY wOWQxNzY1Y2MyMjkxNTM0ZDE5YWQ2NDE3ZjcxMmU1ZDZlMDliZDY0ODcyNTA 5ODA5MTBiZmU1NDc4MmZkNTZkYjcJM2UJMDYJMTgJYzYJMjYJMmEJOWIJZjIJN mYJNTcJNTMJNzYJYTIJYmQNCmVlCTE0CTUyCWU4CWM3CTE1CWU4CWQ3CWQ wCTQ0CTFhCTEzCWUwCWFlCTJlCTNhDQpjMQk3MQk5OQkxNwljNQk5OAk1Ygk4Y wkxNQk4NAk1MAk5NgkzNwk0YgkzYQk3YQ==\u2033 }", + "p0068": "The example hashing and encryption handler 306 outputs the encrypted and/or hashed key-value pair 320 to the example compression handler 308 .", + "p0069": "The compression handler 308 of the illustrated example determines if compression of the encrypted and/or hashed key-value pair 320 is requested. For example, the request to generate the xJSON file may identify one or more key-value pairs and/or objects for which compression is requested. Alternatively, the compression handler 308 may automatically determine that compression is desired when a key-value pair and/or an object exceeds a threshold size. When compression is requested, the compression handler 308 compresses the encrypted and/or hashed key-value pair 320 to generate the example compressed key-value pair 322 . The example compression handler 308 of the illustrated example inserts a key for the compressed data in the example compressed key-value pair 322 . For example, the compression handler 308 may insert a generated key for the compressed data to ensure that the key for each set of generated data is unique. The example compression handler 308 of FIG. 3 also inserts a compression identifier in the example compressed key-value pair 308 to indicate to the example parser 204 that the compressed key-value pair 308 is compressed. In the illustrated example, the example compression handler 308 inserts a value for the key that identifies metadata for the compression and includes the compressed data. For example, the compression handler 308 may utilize gzip for compressing the encrypted and/or hashed key-value pair 320 and may insert metadata identifying the algorithm and the look-up table for the compression algorithm. For example, the result of compressing the person object may be:", + "p0070": "}", + "p0071": "The example compression handler 308 outputs the compressed key-value pair 322 to the example serialization processor 310 .", + "p0072": "The example serialization processor 310 of the illustrated example determines if serialization of the example compressed key-value pair 322 is requested. For example, the request to generate an xJSON file may identify one or more key-value pairs and/or objects for which serialization is requested. When serialization is requested, the example serialization processor 310 serializes the requested compressed key-value pair to generate an example serialized key-value pair 324 . The serialization processor 310 of the illustrated example inserts a key for the serialized key-value pair 324 . For example, the serialization processor 310 may insert a generated key for the serialized key-value pair 324 to ensure that the key for each set of generated data is unique. The example serialization processor 310 also inserts a serialization identifier in the key to indicate to the example parser 204 that the serialized key-value pair is serialized. The example serialization processor 310 of FIG. 3 inserts the serialized data as the value for the key of the serialized key-value pair 324 . In the illustrated example, the example serialization processor 310 converts the binary data resulting from the serializing to ASCII text using Base64 conversion. For example, the result of serializing the person object may be:", + "p0074": "The example serialization processor 310 of the illustrated example transmits the resulting serialized key-value pair 324 to the example data handler 302 for transmission to the destination for the object notation data 314 . For example, the object notation data 314 i 1 e may be transmitted via the example interface 202 to the example web service 102 for parsing by the example parser 204 in the xJSON handler 110 implemented in the example web service 102 . The example web service 102 may then process the data objects in accordance with the operation of the example web service 102 . Alternatively, the object notation data 314 may be transmitted to any other desired location. In some examples, the data handler 302 includes references to invoke functions in the object notation data 314 . For example, a function referenced as \u201cfunc1 \u201d will cause a parser \u201d will cause the parser to cause \u201cfunc1\u201d to be invoked by the server listening at the location \u201cuni.\u201d", + "p0075": "FIG. 4 is a block diagram of an example implementation of the parser 204 of FIG. 2 . The example parser 204 of FIG. 4 includes a data handler 402 , a string processor 404 , a decryption handler 410 , a deserialization processor 406 , and a decompression handler 408 .", + "p0076": "The data handler 402 of the example of FIG. 4 receives an example object notation data 412 file to be processed. The example data handler 402 extracts the key-value pairs and/or objects from the object notation data 412 and transmits them to the example string processor 404 . For example, the data handler 402 may extract one key-value pair at a time and transmit the key-value pair as the example key-value pair 414 for processing by the example string processor 404 , the example deserialization processor 406 , the example decompression handler 408 , and/or the example decryption handler 410 . Alternatively, the data handler 402 may extract multiple key-value pairs for processing by the example string processor 404 , the example deserialization processor 406 , and the example decompression handler 408 , and the example decryption handler 410 .", + "p0077": "Following processing by one or more of the example string processor 404 , the example deserialization processor 406 , and the example decompression handler 408 , and the example decryption handler 410 , the example data handler 402 receives the data object and transmits example object data 422 containing the objects extracted from the example object notation data 412 to the example JavaScript interpreter 208 .", + "p0078": "The example string processor 404 of FIG. 4 receives the example key-value pair 414 extracted from the object notation data 412 by the data handler 402 and automatically determines if the example key-value pair 414 is an xJSON file or a JSON file based on the presence or lack of string literals . According to the illustrated example, the string processor 404 determines that the example key-value pair 414 is associated with file is a JSON file when the string literals are present and determines that the example key-value pair 414 is an xJSON file when the string literals are not present. The example string processor 404 additionally removes the string literals when they are present to reduce the data size. The example string processor 404 transmits processed key-value pair 416 to the example deserialization processor 406 .", + "p0079": "The example deserialization processor 406 of FIG. 4 determines if the processed key-value pair 416 include a serialization identifier . When the example deserialization processor 406 determines that the serialization identifier is included in the example processed key-value pair 416 , the example deserialization processor 406 deserializes the example processed key-value pair 416 . For example, serialized data may be encoded in Base64 and the example serialization processor 406 will decode the Base64 representation to retrieve the original key-value pair . After performing any needed deserialization, the example deserialization handler 406 of the illustrated example transmits a deserialized key-value pair 417 to the example decompression handler 408 .", + "p0080": "The example decompression handler 408 determines if the example deserialized key-value pair 417 include a compression identifier . When the example decompression handler 408 of this example determines that the compression identifier is included in a key of the example deserialized key-value pair 417 , the example decompression handler 408 decompresses the example deserialized key-value pair 417 . The example decompression handler 408 of the illustrated example retrieves the identity of the compression algorithm from metadata inserted into the value of the example deserialized key-value pair 417 during compression by the example compression handler 308 . For example, the metadata may include an identity of the compression algorithm , parameters for use during the compression and/or decompression , etc. After performing any needed decompression, the example decompression handler 408 transmits a decompressed key-value pair 418 to the example decryption handler 410 .", + "p0081": "The decryption handler 410 of this example determines if the decompressed key-value pair 418 includes an encryption identifier . When the decryption handler 410 of the illustrated example determines that the encryption identifier is included in a key of the example decompressed key-value pair 418 , the decryption handler 410 decrypts the key-value pair. For example, the decryption handler 410 may have access to private keys installed on the device on which the xJSON handler 110 is implemented . The decryption handler 410 of the illustrated example may retrieve the private key corresponding to decompressed key-value pair 418 and utilize the private key for decrypting the decompressed key-value pair 418 . Alternatively, the decryption handler 410 may prompt a user to input a private key for performing the decryption.", + "p0082": "The example decryption handler 410 of FIG. 4 determines the appropriate private key for the decryption by analyzing the keys field inserted into the decompressed key-value pair 418 and/or the example object notation data 412 . Alternatively, information identifying the keys utilized for encrypting and/or decrypting the decompressed key-value pair 418 may be stored in any other location . In some examples where multiple keys are utilized in the example object notation data 412 , the encryption identifier may include an identifier for the particular one of the keys utilized for encrypting the decompressed key-value pair 418 . For example, as described above in conjunction with the hashing and encryption handler 306 , the encryption identifier may be \u201c \u201d where X is an index value identifying one of the keys in the keys field inserted in the object notation data 412 .", + "p0083": "After performing any needed decryption, the example decryption handler 410 of the illustrated example transmits an example decrypted key-value pair 420 to the data handler 402 for transmission of the example object data 422 to the example JavaScript interpreter 208 .", + "p0084": "While an example manner of implementing the generator 206 of FIG. 2 is illustrated in FIG. 3 , one or more of the elements, processes and/or devices illustrated in FIG. 3 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example data handler 302 , the example string processor 304 , the example hashing and encryption handler 306 , the example compression handler 308 , the example serialization processor 310 , and/or, more generally, the generator 206 of FIGS. 2 and 3 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example data handler 302 , the example string processor 304 , the example hashing and encryption handler 306 , the example compression handler 308 , and/or the example serialization processor 310 could be implemented by one or more analog or digital circuit , logic circuits, programmable processor , application specific integrated circuit ), programmable logic device ) and/or field programmable logic device ). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example data handler 302 , the example string processor 304 , the example hashing and encryption handler 306 , the example compression handler 308 , and/or the example serialization processor 310 is/are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk , a compact disk , a Blu-ray disk, etc. storing the software and/or firmware. Further still, the xJSON handler 110 of FIG. 1 and/or the generator 206 of FIGS. 2 and/or 3 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIG. 3 , and/or may include more than one of any or all of the illustrated elements, processes and devices.", + "p0085": "Flowcharts representative of example machine readable instructions for implementing the example generator 206 are shown in FIGS. 5-8 . In these examples, the machine readable instructions comprise program for execution by a processor such as the processor 1212 shown in the example processor platform 1200 discussed below in connection with FIG. 12 . The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk , a Blu-ray disk, or a memory associated with the processor 1212 , but the entire program and/or parts thereof could alternatively be executed by a device other than the processor 1212 and/or embodied in firmware or dedicated hardware. Further, although the example program are described with reference to the flowcharts illustrated in FIGS. 5-8 , many other methods of implementing the example generator 206 may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.", + "p0086": "As mentioned above, the example processes of FIGS. 5-8 may be implemented using coded instructions stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory , a compact disk , a digital versatile disk , a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration . As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and transmission media. As used herein, \u201ctangible computer readable storage medium\u201d and \u201ctangible machine readable storage medium\u201d are used interchangeably. Additionally or alternatively, the example processes of FIGS. 5-8 may be implemented using coded instructions stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration . As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and transmission media. As used herein, when the phrase \u201cat least\u201d is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term \u201ccomprising\u201d is open ended.", + "p0087": "The example computer readable instructions of FIG. 5 begins when the example data handler 302 receives data and a request to generate object notation data . For example, the example data handler 302 may receive a JavaScript object from the example JavaScript interpreter 208 . The example data handler 302 determines if xJSON output is requested . For example, the request to generate the xJSON file may include an indication that an xJSON specific file is requested. An xJSON specific file is a file that does not need to support parsing by devices that do not support xJSON. For example, for files that support parsing by JSON the keys and values are surrounded by quotation marks and for xJSON files that do not need to support JSON the keys and values do not need to be surrounded by quotation marks to reduce the file size. When the request indicates that the output file is to be an xJSON file type, the example data handler 302 inserts a content type identifier indicating that the file is an xJSON file type . When the request indicates that the output file is to be a JSON file type, the example data handler 302 inserts a content type identifier indicating that the file is a JSON file type .", + "p0088": "After the content type is set in block 506 or block 508 , the example data handler 302 selects a data object . For example, the example data handler 302 may select the firstName element of the example person object described above. The example data handler 302 then generates a key-value pair for the selected element . For example, the example data handler 302 may create a key named \u201cfirstName\u201d and a value containing the value for the firstName element to generate the JSON key-value pair: \u201cfirstName: John.\u201d", + "p0089": "The example string processor 304 then determines if the content type was set to xJSON for the file . If the content type was not set to xJSON , the example string processor 304 inserts quotation marks around the key and the value in the generated key-value pair .", + "p0090": "After the string processor 304 inserts the quotation marks in block 516 or after the string processor 304 determines that the content type for the file is set to xJSON , the example hashing and encryption handler 306 determines if the key-value pair is to be hashed and/or encrypted . The example hashing and encryption handler 306 may determine that the key-value pair is to be hashed and/or encrypted when the request to generate the xJSON file indicates that the key-value pair is to be hashed and/or encrypted. Alternatively, the hashing and encryption handler 306 may automatically determine that data is to be encrypted when detecting that the key-value pair contains sensitive data . When the hashing and encryption handler 306 determines that the key-value pair is to be hashed and/or encrypted, the example hashing and encryption handler 306 hashes and/or encrypts the key-value value pair . Example computer readable instructions for hashing and/or encrypting the key-value pair are described in conjunction with FIG. 8 .", + "p0091": "After the example hashing and encryption handler 306 determines that hashing and encryption are not requested or the hashing and encryption handler hashes and/or encrypts the key-value pair , the example compression handler 308 determines if the key-value pair is to be compressed . The compression handler 308 may determine that the key-value pair is to be compressed when the request to generate the object notation data indicates that the key-value pair is to be compressed. Alternatively, the example compression handler 308 may determine that the key-value pair is to be compressed when the size of the value exceeds a threshold level. When the compression handler 308 determines that the key-value pair is to be compressed, the example compression handler 308 compresses the key-value value pair . Example computer readable instructions for compressing the key-value pair are described in conjunction with FIG. 6 .", + "p0092": "After the example compression handler 308 determines compression is not requested or the compression handler 308 compresses the key-value pair , the example serialization processor 310 determines if the key-value pair is to be serialized . The example serialization processor 310 may determine that the key-value pair is to be serialized when the request to generate the object notation data indicates that the key-value pair is to be serialized. When the serialization processor 310 determines that the key-value pair is to be serialized, the example serialization processor 310 serializes the key-value value pair . An example process for serializing the key-value pair is described in conjunction with FIG. 7 .", + "p0093": "After performing any requested hashing and/or encrypting , compressing , and serializing , the example data handler 302 inserts the generated key-value pair in the object notation data . The example data handler 302 determines if there are additional data objects and/or elements for which key-value pairs are to be generated . When there are additional objects and/or elements for key-value pair generation, control returns to block 510 to process the next object and/or element. When there are no additional objects and/or elements for key-value pair generation the example computer readable instructions of FIG. 5 end.", + "p0094": "FIG. 6 is a flowchart of example computer readable instructions to compress a key-value pair. The example computer readable instructions of FIG. 6 may be used to implement block 524 of FIG. 5 . The example computer readable instructions of FIG. 6 begins when the example compression handler 308 determines a compression algorithm . For example, a request to compress a key-value pair may specify a compression algorithm to be used. Alternatively, the example compression handler 308 may include a default compression algorithm . The example compression handler 308 then compresses the key-value pair using the determined compression algorithm . The example compression handler 308 then inserts a compression identifier in the key for the key-value pair . For example, the compression identifier may be any indication that may indicate that the key-value pair is compressed . For example, a compressed key-value pair may include a key placeholder and the compression identifier . The example compression handler 308 then inserts metadata regarding the compression in the value for the key-value pair . For example, the compression handler 308 may insert an identification of the algorithm used for compression and parameters for the compression . The example computer readable instructions of FIG. 6 then end. For example, control may return to block 526 of FIG. 5 .", + "p0095": "FIG. 7 is a flowchart of example computer readable instructions that may be executed to serialize a key-value pair. The example computer readable instructions of FIG. 7 may be used to implement block 528 of FIG. 5 . The example computer readable instructions of FIG. 7 begins when the example serialization processor 310 determines a serialized value for the value in the key-value pair . For example, the serialization processor 310 may serialize the value of the key-value pair and perform a binary to text conversion to store the serialized data in the object notation data. The example serialization processor 310 then modifies the key of the key-value pair to insert a serialization identifier in the key . The example computer readable instructions of FIG. 7 then end. For example, control may return to block 530 of FIG. 5 .", + "p0096": "FIG. 8 is a flowchart of example computer readable instructions to hash and/or encrypt a key-value pair. The example computer readable instructions of FIG. 8 may be used to implement block 520 of FIG. 5 . The process of FIG. 8 begins when the example hashing and encryption handler 310 of the illustrated example determines a cipher and a key to be used . For example a request to hash and/or encrypt may include an identification of a cipher and/or a key that is to be used. Alternatively, the hashing and encryption handler 310 may utilize a default cipher and/or private key. The example hashing and encryption handler 310 then packs the string to be encrypted . For example, the example hashing and encryption handler 310 packs the key-value pair by removing any quotation marks. The hashing and encryption handler 310 may perform any other packing to remove any other characters. The example hashing and encryption handler 310 then signs the key-value pair using the identified key .", + "p0097": "The example hashing and encryption handler 310 then determines if the cipher includes encryption . For example, the cipher may be a cipher that only includes hashing or may be a cipher that includes hashing and encryption. When the cipher does not include encryption, control proceeds to block 822 for hashing of the key-value pair. When the cipher includes encryption, the hashing and encryption handler 310 encrypts the signed key-value pair . The example hashing and encryption handler 310 then converts the encrypted value to a string for insertion in the xJSON file . The example hashing and encryption handler 310 transfers the encrypted value to a string using Base64 encoding.", + "p0098": "After encrypting the key-value pair , the hashing and encryption handler 310 inserts an encryption identifier in the key of the key-value pair . The example hashing and encryption handler 310 then inserts the metadata identifying the cipher in the xJSON file . For example, the cipher metadata may be inserted in a key-value pair with key name \u201ckeys.\u201d The example hashing and encryption handler 310 then determines if there are multiple ciphers in the keys metadata . If there are multiple ciphers in the keys metadata, the example hashing and encryption handler 310 inserts a cipher identifier in the key of the encrypted key-value pair . For example, the hashing and encryption handler 310 may insert an index corresponding to the cipher in the keys metadata .", + "p0099": "After the hashing and encryption handler 310 has determined that the cipher does not include encryption , has determined that there are not multiple ciphers , or has inserted the identifier of the cipher in the key , the example hashing and encryption handler 310 determines a hash for the value of the key-value pair . For example, the hash may be determined using a double Pearson hashing. The example hashing and encryption handler 310 inserts the value of the hash into the key for the key-value pair . For example, the hash value may be inserted following a hashing identifier ). For example, the hash may be inserted as . A key for a value that is encrypted and hashed may be \u201c \u201d where a single cipher is present and \u201c \u201d where there are multiple ciphers and the first cipher was utilized for the encryption.", + "p0100": "The example computer readable instructions of FIG. 8 then end. For example, control may return to block 522 of FIG. 5 .", + "p0101": "While an example manner of implementing the parser 204 of FIG. 2 is illustrated in FIG. 4 , one or more of the elements, processes and/or devices illustrated in FIG. 4 may be combined, divided, re-arranged, omitted, eliminated and/or implemented in any other way. Further, the example data handler 402 , the example string processor 404 , the example deserialization processor 406 , the example decompression handler 408 , the example decryption handler 410 , and/or, more generally, the parser 204 of FIGS. 2 and 4 may be implemented by hardware, software, firmware and/or any combination of hardware, software and/or firmware. Thus, for example, any of the example data handler 402 , the example string processor 404 , the example deserialization processor 406 , the example decompression handler 408 , and/or the example decryption handler 410 could be implemented by one or more analog or digital circuit , logic circuits, programmable processor , application specific integrated circuit ), programmable logic device ) and/or field programmable logic device ). When reading any of the apparatus or system claims of this patent to cover a purely software and/or firmware implementation, at least one of the example data handler 402 , the example string processor 404 , the example deserialization processor 406 , the example decompression handler 408 , and/or the example decryption handler 410 is/are hereby expressly defined to include a tangible computer readable storage device or storage disk such as a memory, a digital versatile disk , a compact disk , a Blu-ray disk, etc. storing the software and/or firmware. Further still, the xJSON handler 110 of FIG. 1 and/or the parser 204 of FIGS. 2 and/or 4 may include one or more elements, processes and/or devices in addition to, or instead of, those illustrated in FIG. 4 , and/or may include more than one of any or all of the illustrated elements, processes and devices.", + "p0102": "Flowcharts representative of example machine readable instructions for implementing the example parser 204 are shown in FIGS. 9-11 . In these examples, the machine readable instructions comprise program for execution by a processor such as the processor 1212 shown in the example processor platform 1200 discussed below in connection with FIG. 12 . The program may be embodied in software stored on a tangible computer readable storage medium such as a CD-ROM, a floppy disk, a hard drive, a digital versatile disk , a Blu-ray disk, or a memory associated with the processor 1212 , but the entire program and/or parts thereof could alternatively be executed by a device other than the processor 1212 and/or embodied in firmware or dedicated hardware. Further, although the example program are described with reference to the flowcharts illustrated in FIGS. 9-11 , many other methods of implementing the example parser 204 may alternatively be used. For example, the order of execution of the blocks may be changed, and/or some of the blocks described may be changed, eliminated, or combined.", + "p0103": "As mentioned above, the example processes of FIGS. 9-11 may be implemented using coded instructions stored on a tangible computer readable storage medium such as a hard disk drive, a flash memory, a read-only memory , a compact disk , a digital versatile disk , a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration . As used herein, the term tangible computer readable storage medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and transmission media. As used herein, \u201ctangible computer readable storage medium\u201d and \u201ctangible machine readable storage medium\u201d are used interchangeably. Additionally or alternatively, the example processes of FIGS. 9-11 may be implemented using coded instructions stored on a non-transitory computer and/or machine readable medium such as a hard disk drive, a flash memory, a read-only memory, a compact disk, a digital versatile disk, a cache, a random-access memory and/or any other storage device or storage disk in which information is stored for any duration . As used herein, the term non-transitory computer readable medium is expressly defined to include any type of computer readable storage device and/or storage disk and to exclude propagating signals and transmission media. As used herein, when the phrase \u201cat least\u201d is used as the transition term in a preamble of a claim, it is open-ended in the same manner as the term \u201ccomprising\u201d is open ended.", + "p0104": "FIG. 9 is a flowchart of example computer readable instructions for the example parser 204 to parse object notation data . The example computer readable instructions of FIG. 9 begin when the example data handler 402 receives object notation data . For example the data handler 402 or another component of a device including the xJSON handler 110 may request data from another device that transmits data as xJSON data.", + "p0105": "In some examples, the data handler 402 of the illustrated example requests only a portion of an available object notation data. For example, an example xJSON file might include 100,000 key value pairs, which would exhaust the memory of a low power device attempting to parse the xJSON file. Accordingly, the example data handler 402 requests a desired portion . For example, the data handler 402 may reference a particular portion of the object notation data using dot notation . Thus, the example data handler 402 may retrieve a desired key and/or object of interest without the need to retrieve the entire object notation data. In an example implementation, an object may be referenced as \u201c# object\u201d where \u201cobject\u201d is the name of the object, \u201c@uri\u201d where \u201curi\u201d is the location from which the object notation data may be retrieved, and @uri#object.subobject where \u201csubobject\u201d identifies an object and/or key within the object \u201cobject\u201d in the object notation data location at \u201curi.\u201d", + "p0106": "The example data handler 402 selects the first key-value pair in the object notation data . The example string processor 404 determines if the key-value pair includes string literals . When the key-value pair does not include string literals, the example string processor 404 determines that the received file is of the xJSON type and stores an indication that the file is an xJSON file . Control then proceeds to block 922 .", + "p0107": "When the string processor 404 determines that the key-value pair includes string literals, the string processor 404 stores the type as JSON . For example, the file may be a JSON compatible file because it includes the string literals, but the file may include xJSON extensions. The example string processor 404 then removes the quotation marks from the key-value pair to reduce the size of the xJSON file .", + "p0108": "After the string processor 404 sets the type as xJSON or after the string processor 404 removes the quotation marks , the example deserialization processor 406 determines if the key includes a serialization identifier . When the key includes a serialization identifier, the example deserialization processor 406 deserializes/demarshalls the serialized data .", + "p0109": "When the key-value pair does not include a serialization identifier or after deserialization of the key-value pair , the example decompression handler 408 determines if the key includes a compression identifier . When the key includes a compression identifier, the example decompression handler 408 decompresses the key-value pair . Example computer readable instructions that may be executed to decompress a key-value pair is described in conjunction with FIG. 11 .", + "p0110": "When the key-value pair does not include a compression identifier identifier or after decompression of the key-value pair , the example decryption handler 410 determines if the key of the key-value pair includes an encryption identifier . When the key includes the encryption identifier, the decryption handler 410 decrypts the key-value pair . Example computer readable instructions to decrypt the key-value pair are described in conjunction with FIG. 10 .", + "p0111": "FIG. 10 is a flowchart of example computer readable instructions to decrypt and encrypted key-value pair. The example computer readable instructions may be used to implement block 924 of FIG. 9 . The example computer readable instructions begin when the decryption handler 410 determines the cipher and key utilized during encryption of the key-value pair . The example decryption handler 410 determines the cipher and public key from the keys metadata included in object notation data. In some examples, the decryption handler 410 selects the cipher and key from a list of keys using an index identified in the encryption handler.", + "p0112": "The example decryption handler 410 then obtains the private key corresponding to the public key used during encryption . For example, the private key may be stored in a set of private keys stored in the parser 204 . Alternatively, the decryption handler 410 may display a prompt requesting that a user provide a private key corresponding to an identified public key. The decryption handler 410 then decrypts the encrypted data using the private key and the identified cipher . The example computer readable instructions of FIG. 10 then end. For example, control may return to block 926 of FIG. 9 .", + "p0113": "FIG. 11 is a flowchart of example computer readable instructions to decompress a key-value pair. The example computer readable instructions of FIG. 11 may be used to implement block 920 of FIG. 9 . The example computer readable instructions of FIG. 11 begin when the example decompression handler 408 determines a compression algorithm that was utilized to compress the key-value pair . For example, the decompression handler 408 determines the compression algorithm from the metadata inserted in value of the compressed key-value pair. The example decompression handler 408 then determines parameters for the compression . For example, the decompression handler 408 may extract the parameters from metadata inserted in the value of the key-value pair. For example, the parameters may include a look-up table utilized by the compression algorithm. The example decompression handler 408 then decrypts the key-value pair using the identified compression algorithm and the parameters . The example computer readable instructions of FIG. 11 then end. For example, control may return to block 922 of FIG. 9 .", + "p0114": "FIG. 12 is a block diagram of an example processor platform 1200 structured to execute the instructions of FIGS. 5, 6, 7, 8, 9, 10 , and/or 11 to implement the example first device 106 and/or the example web service 102 including the example interface 202 , the example parser 204 , the example generator 206 , and/or the example JavaScript interpreter 210 . The processor platform 1200 can be, for example, a personal computer, a mobile device , a personal digital assistant , or any other type of computing device capable of processing images.", + "p0115": "The processor platform 1200 of the illustrated example includes a processor 1212 . The processor 1212 of the illustrated example is hardware. For example, the processor 1212 can be implemented by one or more integrated circuits, logic circuits, microprocessors or controllers from any desired family or manufacturer. The example processor 1212 of FIG. 12 may implement the components of the example xJSON handler 110 including the example parser 204 , the example generator 206 , and the example JavaScript interpreter 208 to parse and generate xJSON files and data.", + "p0116": "The processor 1212 of the illustrated example includes a local memory 1213 . The processor 1212 of the illustrated example is in communication with a main memory including a volatile memory 1214 and a non-volatile memory 1216 via a bus 1218 . The volatile memory 1214 may be implemented by Synchronous Dynamic Random Access Memory , Dynamic Random Access Memory , RAMBUS Dynamic Random Access Memory and/or any other type of random access memory device. The non-volatile memory 1216 may be implemented by flash memory and/or any other desired type of memory device. Access to the main memory 1214 , 1216 is controlled by a memory controller.", + "p0117": "The processor platform 1200 of the illustrated example also includes an interface circuit 1220 . The interface circuit 1220 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus , and/or a PCI express interface. The example interface circuit may implement the example interface 202 of the xJSON handler 110 of FIGS. 1 and/or 2 to interface the processor platform 1200 with the example network 104 of FIG. 1 .", + "p0118": "In the illustrated example, one or more input devices 1222 are connected to the interface circuit 1220 . The input device 1222 permit a user to enter data and commands into the processor 1212 . The input device can be implemented by, for example, an audio sensor, a microphone, a camera , a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, isopoint and/or a voice recognition system.", + "p0119": "One or more output devices 1224 are also connected to the interface circuit 1220 of the illustrated example. The output devices 1224 can be implemented, for example, by display devices , an organic light emitting diode , a liquid crystal display, a cathode ray tube display , a touchscreen, a tactile output device, a light emitting diode , a printer and/or speakers). The interface circuit 1220 of the illustrated example, thus, typically includes a graphics driver card, a graphics driver chip or a graphics driver processor.", + "p0120": "The interface circuit 1220 of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem and/or network interface card to facilitate exchange of data with external machines via a network 1226 , a telephone line, coaxial cable, a cellular telephone system, etc.).", + "p0121": "The processor platform 1200 of the illustrated example also includes one or more mass storage devices 1228 for storing software and/or data. Examples of such mass storage devices 1228 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, RAID systems, and digital versatile disk drives.", + "p0122": "The coded instructions 1232 of FIGS. 5, 6, 7, 8, 9, 10 , and/or 11 may be stored in the mass storage device 1228 , in the volatile memory 1214 , in the non-volatile memory 1216 , and/or on a removable tangible computer readable storage medium such as a CD or DVD.", + "p0123": "Examples disclosed herein provide extensions to object notation data . In some examples, usage of data storage and communication bandwidth are reduced used by packing and/or compressing portions of the object notation data. In some examples, computer processing resource usage is reduced by allowing portions of object notation data to be packed/compressed, serialized, and/or compressed while allowing other portions of the object notation data not to be extended. For example, in a JSON file, using examples disclosed herein, a single key-value pair can be encrypted without requiring the entire JSON file to be encrypted, which reduces the amount of processing required to encrypt and decrypt the elements of the JSON file. In some examples, backward compatibility with devices that do not support the extension is provided by generating output that follow grammar rules set by prior object notation protocols. Accordingly, such extended files that meet the grammar rules of the prior protocol will not trigger errors when parsing the extended files with a device that does not support the extensions.", + "p0124": "Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.", + "features": { + "F1": "A process of producing an extended JavaScript Notation file involves determining a cipher and a key to use in at least one of hashing and encrypting a JavaScript Object Notation key-value pair.", + "F2": "The key-value pair is signed using the determined key.", + "F3": "The signed key-value pair is encrypted when the cipher includes encryption.", + "F4": "The encrypted key-value pair is converted to a string for insertion in an extended JavaScript Object Notation file.", + "F5": "An encryption identifier is inserted in the key of the key-value pair.", + "F6": "Metadata identifying the cipher is inserted in the xJSON file.", + "F7": "A hash for the value of the key-value pair is determined and the key-value pair is hashed.", + "F8": "The value of the hash is inserted into the key for the key-value pair.", + "F9": "Determining a cipher includes determining whether the cipher includes encryption, hashing, or hashing and encryption.", + "F10": "When the cipher does not include encryption, proceeding to determine a hash for the value of the key-value pair.", + "F11": "Converting includes transferring the encrypted value to a string using Base64 encoding.", + "F12": "Inserting an encryption identifier in the key of the key-value pair includes inserting an identifier \u201c \u201d in the key.", + "F13": "Inserting metadata includes inserting the metadata in a key-value pair with the key name \u201ckeys\u201d.", + "F14": "Determining whether there are multiple ciphers in the keys metadata.", + "F15": "If there are multiple ciphers in the keys metadata, a cipher identifier is inserted in the key of the encrypted key-value pair.", + "F16": "Inserting metadata includes inserting an index corresponding to the cipher in the keys metadata.", + "F17": "Inserting metadata includes inserting an index \u201c \u201d corresponding to the cipher in the keys metadata, where the cipher is the second cipher in the keys metadata.", + "F18": "Determining a hash for the value of the key-value pair and hashing the key-value pair occurs after determining that the cipher does not include encryption, after determining that there are not multiple ciphers, or after inserting the identifier of the cipher in the key.", + "F19": "Receiving a request to hash and/or encrypt includes an identification of a cipher, a key, or a private key.", + "F20": "Extended JavaScript Object Notation generator circuitry comprises data handler circuitry with a first input receiving object data, a second input, and a file output.", + "F21": "The data handler circuitry records the content type for an extended JavaScript Object Notation file, creates a key-value pair for the object data by creating a key named for the variable of the object data and creating a corresponding value for the value of the variable, and sends the key-value pair to the output.", + "F22": "String processor circuitry has an input coupled to the file output and an output, processing the object notation data to be parsed by xJSON capable devices.", + "F23": "Hashing and encryption handler circuitry has an input coupled to the output of the string processor circuitry and an output coupled to the second input of the data handler circuitry.", + "F24": "The hashing and encryption handler circuitry determines a cipher and a key to be used in at least one of hashing and encrypting a JavaScript Object Notation key-value pair.", + "F25": "The key-value pair is signed using the determined key by the hashing and encryption handler circuitry.", + "F26": "The signed key-value pair is encrypted when the cipher includes encryption by the hashing and encryption handler circuitry.", + "F27": "The encrypted key-value pair is converted to a string for insertion in an extended JavaScript Object Notation file by the hashing and encryption handler circuitry.", + "F28": "The encryption identifier is inserted in the key of the key-value pair by the hashing and encryption handler circuitry.", + "F29": "Metadata identifying the cipher is inserted in the xJSON file by the hashing and encryption handler circuitry.", + "F30": "A hash for the value of the key-value pair is determined and the key-value pair is hashed by the hashing and encryption handler circuitry.", + "F31": "The value of the hash is inserted into the key for the key-value pair by the hashing and encryption handler circuitry.", + "F32": "The hashing and encryption handler circuitry determines whether the cipher includes encryption, hashing, or hashing and encryption.", + "F33": "When the cipher does not include encryption, the hashing and encryption handler circuitry proceeds to determining a hash for the value of the key-value pair.", + "F34": "The hashing and encryption handler circuitry transfers the encrypted value to a string using Base64 encoding.", + "F35": "The hashing and encryption handler circuitry inserts an encryption identifier in the key of the key-value pair by inserting an identifier \u201c \u201d in the key.", + "F36": "The hashing and encryption handler circuitry inserts the metadata in a key-value pair with the key name \u201ckeys\u201d.", + "F37": "The hashing and encryption handler circuitry determines whether there are multiple ciphers in the keys metadata.", + "F38": "If there are multiple ciphers in the keys metadata, the hashing and encryption handler circuitry inserts a cipher identifier in the key of the encrypted key-value pair." + } + } + }, + { + "FAN": "79482665", + "CPC": [ + "G06F-016/00", + "G06F-016/285", + "G06F-016/583", + "G06F-018/214", + "G06F-018/2411", + "G06F-018/2451", + "G06K-009/00", + "G06K-009/00/275", + "G06K-009/00/288", + "G06K-009/00/335", + "G06K-009/62", + "G06K-2209/09", + "G06Q-010/10", + "G06Q-050/26", + "G06Q-050/26/5", + "G06V-020/40", + "G06V-020/44", + "G06V-020/46", + "G06V-040/16", + "G06V-040/169", + "G06V-040/172", + "G06V-040/20", + "G06V-2201/05", + "G07C-009/00" + ], + "Content": { + "title": "Security check system and method for configuring security check device", + "pa01": "A security inspection system and a method of configuring a security inspection device are provided. In an embodiment, the security inspection system may include: an identity information entry device configured to enter an identification of an inspected person; a parameter determination device configured to determine a parameter for performing a security inspection on the inspected person based on a security factor of the inspected person determined according to user data corresponding to the identification of the inspected person; and a security inspection device configured to perform the security inspection on the inspected person based on the determined parameter. According to embodiments, it is possible to accurately predict the user's behavior and evaluate the risk or potential danger from the user by analyzing and mining the user's comprehensive data, and thus to provide a more accurate security inspection solution.", + "c-en-0001": "A security inspection system, comprising: an identity information entry device configured to enter an identification of an inspected person; a security factor determination device configured to grasp, from a third-party platform, user data related to the inspected person in real time based on the identification of the inspected person, and determine the security factor of the inspected person by substituting the user data of the inspected person into a relationship model between user data and security factors; a parameter determination device configured to determine a parameter for performing a security inspection on the inspected person based on a security factor of the inspected person determined according to user data corresponding to the identification of the inspected person, the parameter comprising one or more of a classifier type, a classifier parameter and an alarm threshold in a trained hidden object recognition algorithm determined based on the security factor; and a security inspection device configured to perform the security inspection on the inspected person, wherein performing the security inspection on the inspected person comprises processing a scanning image of the inspected person based on the determined parameter for performing the security inspection on the inspected person, so as to identify whether or not the inspected person has concealed some article, wherein the security inspection device is further configured to perform the security inspection on the inspected person by using the trained hidden object recognition algorithm based on the determined parameter, comprising: extracting features of the inspected person; calculating projected vectors of the features in a dictionary; and inputting the projected vectors into a classifier corresponding to the determined parameter for classification, wherein the user data of the current inspected person is updated at an end of each security inspection, updating the user data of the inspected person comprises updating the user data of the third-party platform in real time, and the relationship model is established by using the updated user data for offline model training, wherein in the relationship model, user data from the third-party platform are classified into a plurality of categories assigned with different security factors respectively, and a category to which the inspected person belongs is determined according to a Euclidean distance between the user data of the inspected person and a center of each of the plurality of categories; and wherein the security factor comprises a first security factor and a second security factor less than the first security factor, the parameter comprises a first parameter determined based on the first security factor and a second parameter determined based on the second security factor, the first parameter comprises a linear classifier, a first penalty factor of the classifier and a first alarm threshold, the second parameter comprises a nonlinear classifier, a second penalty factor of the classifier and a second alarm threshold, the generalization ability of the linear classifier is higher than the generalization ability of the nonlinear classifier, the second penalty factor is higher than the first penalty factor, and the second alarm threshold is lower than the first alarm threshold.", + "c-en-0002": "The security inspection system according to claim 1 , wherein the identity information entry device is further configured to obtain a registered photo of the inspected person.", + "c-en-0003": "The security inspection system according to claim 2 , further comprising: an authentication device configured to extract a face image of the inspected person from the image captured by the video device, and to verify the identity of the inspected person by comparing the extracted face image with a face image in the registered photo and/or a database of untrusted persons.", + "c-en-0004": "The security inspection system according to claim 1 , further comprising: a video device configured to capture an image of the inspected person in real time.", + "c-en-0005": "The security inspection system according to claim 4 , further comprising: an abnormal behavior determination device configured to determine whether the inspected person has an abnormal behavior according to the image of the inspected person captured by the video device.", + "c-en-0006": "The security inspection system according to claim 1 , wherein the user data comprises one or more of personal data, credit, social relationship, or historical behavior of the inspected person.", + "c-en-0007": "The security inspection system according to claim 1 , further comprising: a display device configured to display a corresponding security level of the inspected person during the security inspection.", + "c-en-0008": "A method of configuring a security inspection device in a security inspection system, comprising: acquiring, by an identity information entry device, an identification of an inspected person; grasping, from a third-party platform, user data related to the inspected person in real time based on the identification of the inspected person, and determining the security factor of the inspected person by substituting the user data of the inspected person into a relationship model between user data and security factors; acquiring user data related to the inspected person based on the identification of the inspected person, and determining a security factor of the inspected person according to the acquired user data; determining, by a parameter determination device, a parameter for performing a security inspection on the inspected person based on the security factor of the inspected person, comprising determining one or more of a classifier type, a classifier parameter and an alarm threshold in a trained hidden object recognition algorithm based on the security factor; and configuring the security inspection device based on the determined parameter, wherein the security inspection device is configured to perform the security inspection on the inspected person, wherein the security inspection device is further configured to perform the security inspection on the inspected person by using the trained hidden object recognition algorithm based on the determined parameter, comprising: extracting features of the inspected person; calculating projected vectors of the features in a dictionary; and inputting the projected vectors into a classifier corresponding to the determined parameter for classification, wherein the user data of the current inspected person is updated at an end of each security inspection, updating the user data of the inspected person comprises updating the user data of the third-party platform in real time, and the relationship model is established by using the updated user data for offline model training, wherein performing the security inspection on the inspected person comprises processing a scanning image of the inspected person based on the determined parameter for performing the security inspection on the inspected person, so as to identify whether or not the inspected person has concealed some article, wherein in the relationship model, user data from the third-party platform are classified into a plurality of categories assigned with different security factors respectively, and a category to which the inspected person belongs is determined according to a Euclidean distance between the user data of the inspected person and a center of each of the plurality of categories; and wherein the security factor comprises a first security factor and a second security factor less than the first security factor, the parameter comprises a first parameter determined based on the first security factor and a second parameter determined based on the second security factor, the first parameter comprises a linear classifier, a first penalty factor of the classifier and a first alarm threshold, the second parameter comprises a nonlinear classifier, a second penalty factor of the classifier and a second alarm threshold, the generalization ability of the linear classifier is higher than the generalization ability of the nonlinear classifier, the second penalty factor is higher than the first penalty factor, and the second alarm threshold is lower than the first alarm threshold.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATION", + "p0002": "This application is a U.S. National Stage Filing under 35 U.S.C. 371 from International Application No. PCT/CN2017/095118, filed on Jul. 31, 2017, and published as WO2018/072520 on Apr. 26, 2018, which claims priority to a Chinese Patent Application No. 201610901778.9, filed on Oct. 17, 2016 and entitled \u201cSecurity inspection system and method of configuring security inspection device\u201d, the benefit of priority of each of which is hereby claimed herein, and which applications and publication are hereby incorporated herein by reference in their entirety.", + "p0003": "TECHNICAL FIELD", + "p0004": "The present disclosure relates to the field of security inspection, and in particular, to a security inspection system and a method of configuring security inspection device, enabling a customized security inspection policy based on big data.", + "p0005": "BACKGROUND", + "p0006": "In recent years, terrorist attacks have gradually escalated and expanded on a global scale, seriously threatening personal safety and even national security. The traditional human body security inspection uses metal detection or manual search, which is not only time-consuming but also infringes on personal privacy. With the development of the society and the science and technology, people's requirements for security inspection are increasing. The traditional low-efficiency manual security inspection methods can no longer meet the needs of people in the new era for security inspection. Human body security inspection devices scan the human body in a contactless manner and can detect objects hidden in clothes. Not only metal substances, but also non-metallic substances carried by the people, such as ceramics, plastics, powders, liquids, colloids, or the like, can be detected. This kind of detection is contactless inspection, and the detection result does not show any physical details of the body, enabling adequate protection of the personal privacy of the inspected person. In addition, this kind of detection is more efficient, and can work continuously without interruption, resulting in efficiency much higher than the traditional manual security inspection. This kind of devices can adopt automatic interpretation method to accurately locate suspicious objects, effectively reducing the influence of artificial factors and reducing the labor intensity of security personnel.", + "p0007": "The human body security inspection devices are widely used in the security inspection field because of their unique advantages. However, the existing devices are relatively simple in judging the potential danger of users by only analyzing a scanned image by a hidden object recognition algorithm to determine the possible threat of the inspected person. Also, the hidden object recognition algorithms in the current human body security inspection devices adopt a general algorithm framework, without considering the identity of the users and differences between different users. For all inspected persons, the same type of algorithm frameworks is used for analysis and processing. All of these algorithms are based on the assumption that all users have the same probability of carrying dangerous goods. Therefore, the recognition algorithms are accompanied with false negatives and false positives, which affect the users' experiences.", + "p0008": "SUMMARY", + "p0009": "In view of the above, the present disclosure aims to provide, among others, a security inspection system and a method of configuring a security inspection device, in which security inspection policies can be customized to different users based on user data.", + "p0010": "According to an aspect of the present disclosure, there is provided a security inspection system, comprising: an identity information entry device configured to enter an identification of an inspected person; a parameter determination device configured to determine a parameter for performing a security inspection on the inspected person based on a security factor of the inspected person determined according to user data corresponding to the identification of the inspected person; and a security inspection device configured to perform the security inspection on the inspected person based on the determined parameter.", + "p0011": "According to an embodiment of the present disclosure, the security inspection system may further include a security factor determination device configured to acquire user data related to the inspected person based on the identification of the inspected person, and to determine the security factor of the inspected person based on the acquired user data. For example, the security factor determination device may be configured to determine the security factor of the inspected person by substituting the user data of the inspected person into a relationship model between user data and security factors, wherein in the relationship model, the user data are classified into several categories and each category is assigned a different security factor.", + "p0012": "According to an embodiment of the present disclosure, the identity information entry device may be further configured to obtain a registered photo of the inspected person.", + "p0013": "According to an embodiment of the present disclosure, the security inspection system may further include a video device configured to capture an image of the inspected person in real time.", + "p0014": "According to an embodiment of the present disclosure, the security inspection system may further include an authentication device configured to extract a face image of the inspected person from the image captured by the video device, and to verify the identity of the inspected person by comparing the extracted face image with a face image in the registered photo and/or a database of untrusted persons.", + "p0015": "According to an embodiment of the present disclosure, the security inspection system may further include an abnormal behavior determination device configured to determine whether the inspected person has an abnormal behavior according to the image of the inspected person captured by the video device.", + "p0016": "According to an embodiment of the present disclosure, the user data may comprise one or more of personal data, credit, social relationship, or historical behavior of the inspected person.", + "p0017": "According to an embodiment of the present disclosure, the parameter determination device may be configured to determine a parameter applied in a hidden object recognition algorithm employed by the security inspection device based on the security factor of the inspected person.", + "p0018": "According to an embodiment of the present disclosure, the security inspection system may further include a display device configured to display a corresponding security level of the inspected person during the security inspection.", + "p0019": "According to another aspect of the present disclosure, there is provided a method of configuring the security inspection device in the above security inspection system, comprising: acquiring, by the identity information entry device, an identification of an inspected person; acquiring user data related to the inspected person based on the identification of the inspected person, and determining a security factor of the inspected person according to the acquired user data; determining, by the parameter determination device, a parameter for performing a security inspection on the inspected person based on the security factor of the inspected person; and configuring the security inspection device based on the determined parameter.", + "p0020": "According to an embodiment of the present disclosure, the parameter may comprise a parameter applied in a hidden object recognition algorithm employed by the security inspection device. For example, the parameter may comprise one or more of a classifier type, a classifier parameter, or an alarm threshold.", + "p0021": "According embodiments of the present disclosure, it is possible to extract high-valued information which can characterize user behavior or the like from massive data with low value density by means of data mining or the like, and to establish an individual security factor model based on the data. Based on the model, in combination with the security parameter setting of the security inspection device, it is possible to customize unique security inspection strategies for different users, resulting in more humanized and personalized security inspection while achieving the goal of accurate security inspection.", + "p0022": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0023": "The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the attached drawings, in which:", + "p0024": "FIG. 1 is a schematic diagram showing a security inspection system according to an embodiment of the present disclosure;", + "p0025": "FIG. 2 is a schematic diagram showing various user data;", + "p0026": "FIG. 3 is a schematic diagram showing a flow of a security inspection system according to an embodiment of the present disclosure;", + "p0027": "FIG. 4 is a schematic diagram showing an example clustering of user data.", + "p0028": "DETAILED DESCRIPTION", + "p0029": "Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.", + "p0030": "The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The words \u201ca\u201d, \u201can\u201d, \u201cthe\u201d, or the like, as used herein, shall also include the meaning of \u201ca plurality\u201d and \u201cmultiple\u201d, unless the context clearly dictates otherwise. Furthermore, the terms \u201cinclude\u201d, \u201ccomprise\u201d, or the like as used herein indicate the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.", + "p0031": "All terms used herein have the meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that terms used herein should be interpreted as having a meaning that is consistent with the context of the present description and should not be interpreted in an idealized or overly formal manner.", + "p0032": "Several block diagrams and/or flow charts are shown in the drawings. It should be understood that some of the blocks or combinations thereof in the block diagrams and/or flow charts may be implemented by computer program instructions. Such computer program instructions may be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing apparatus such that the instructions, when executed by the processor, may create means for implementing the functions/operations illustrated in these block diagrams and/or flow charts.", + "p0033": "Accordingly, the techniques of the present disclosure may be implemented in the form of hardware and/or software . Additionally, the techniques of the present disclosure may take the form of a computer program product on a computer readable medium having stored thereon instructions for use by or in connection with an instruction execution system. In the context of the present disclosure, a computer readable medium may be any medium that can contain, store, communicate, propagate, or transport the instructions. For example, the computer readable medium may include but are not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, devices, or propagation medium. Specific examples of the computer readable medium include a magnetic storage device such as a magnetic tape or a hard disk drive , an optical storage device such as a compact disc , a memory such as a random access memory or a flash memory, and/or a wired/wireless communication link.", + "p0034": "FIG. 1 is a schematic diagram showing a security inspection system in accordance with an embodiment of the present disclosure.", + "p0035": "As shown in FIG. 1 , the security inspection system 100 according to this embodiment includes a security inspection device disposed on a security inspection lane 101 .", + "p0036": "The security inspection lane 101 may be located at an entrance to any place where secure access is necessary, such as an airport, station, stadium, museum, or the like. The security inspection lane 101 may be a real lane surrounded by a barrier such as a fence, or a virtual lane defined by, for example, a video device. For example, an inspected person may walk within the surveillance range of the video device, which may be considered a \u201clane.\u201d", + "p0037": "The inspected person passes through the security inspection lane 101 , for example, in a direction indicated by the arrow in the figure, while receiving a security inspection by the security inspection device. The inspected person may be allowed to enter the place to access if he passes the inspection by the security inspection device.", + "p0038": "The security inspection device may include a scanning device 103 and a control device 105 . For human body security inspection, the scanning device 103 may be contactless. For example, the scanning device 103 may illuminate the inspected person by rays , and collect rays from the inspected person , under the control of the control device 105 . The data received by the scanning device 103 may be transmitted to the control device 105 via a wired link or a wireless link , so that a image of the inspected person may be reconstructed by a reconstruction algorithm in the control device 105 . In addition, the control device 105 may also identify objects that may be hidden in the clothing worn by the inspected person by a hidden object recognition algorithm. The imaging results and the recognition results may be output by an output device such as a display, as will be further described below. The control device 105 may include various computing devices such as computers, servers, or the like.", + "p0039": "According to an embodiment of the present disclosure, the hidden object recognition algorithm for the scanned image of the human body may include two parts, i.e., classifier training and online recognition. Both parts are based on partitioned images. The scanned image of the human body may be first partitioned into several areas, such as the arm, the trunk, the leg, or the like, according to positions of some key points of the body. In an example, the classifier training may be performed as follows: 1) creating a database of positive and negative samples for each of the partitioned images; 2) feature extraction: characterizing each of the partitioned images with dense scale-invariant feature transform features; 3) obtaining an over-complete dictionary of dense SIFT features by training with the sparse coding principle; 4) projecting the feature descriptors in 2) into the dictionary to obtain coded vectors; and 5) obtaining a classifier by training with a support vector machine . Online recognition may be performed as follows: 1) first extracting dense SIFT features of the partitioned image to be recognized; 2) calculating projected vectors of the aforementioned features in the dictionary; and 3) inputting the projected vectors into the trained SVM classifier for classification.", + "p0040": "Certainly, in addition to the SVM classifier, other classifiers such as Linear Spatial Pyramid Matching or Locality-constrained Linear coding or a combination thereof may be used. There are many other hidden object recognition algorithms in the art, detailed descriptions of which are omitted here.", + "p0041": "In this example, the scanning device 103 is illustrated as a gate structure, but the disclosure is not limited thereto. The scanning device 103 may have a different form, such as a rotary scan. There are various types of security inspection devices in the field, and currently commonly used human body security inspection devices are based, for example, on the millimeter wave holographic imaging technology.", + "p0042": "In addition, in this example, only the scanning device 103 for the human body is shown. According to an embodiment of the present disclosure, a scanning device for articles such as baggage carried by the inspected person, such as an X-ray scanning device may be further included. For example, when the inspected person enters the security inspection lane 101 , he or she can put his or her baggage onto a conveyor belt to pass through the X-ray scanning device while he or she passes through the scanning device 103 .", + "p0043": "As described above, conventionally the security inspection device employs the same security inspection policy, specifically, the same parameters , for different inspected persons. However, in the actual security inspection process, the possibility of carrying dangerous goods is very different between different persons. If the principle of \u201ceveryone is equal\u201d is still followed during the security inspection of the inspected persons, the accuracy of the security inspection results will be affected.", + "p0044": "According to an embodiment of the present disclosure, different security inspection policies may be employed for different inspected persons. Specifically, a security inspection model specific to each individual may be established in consideration of differences in identity and behavior of the individual from others. As a result, it is possible to significantly improve the efficiency of security inspection, and to reduce false positive results by the security inspection device for trusted persons while strengthening the security inspection intensity for suspicious persons, so as to timely detect and identify potential illegal actions by these people during the security inspection process to prompt policemen to take corresponding measures.", + "p0045": "To this end, the system 100 may include an identity information entry device 107 for entering an identification of the inspected person. The identity information entry device 107 may include various types of information input devices. For example, the identity information entry device 107 may include a keyboard and/or a touch screen. The security personnel may check an identity document of the inspected person such as an identity card or a passport, and manually input through the identity information entry device 107 the identification of the inspected person which may uniquely identify the identity of the inspected person, such as the number on the inspected person's identity document, such as the identity card number, passport number, or the like . Alternatively, the identity information entry device 107 may include an optical or electronic reading device. For example, the identity information entry device 107 may read a one-dimensional code or a two-dimensional code present on the identity document, or may read information from the identity document wirelessly based on, for example, the radio frequency identification or near field communication technology.", + "p0046": "Certainly, it is not necessary to use the identity document, but other documents associated with the user's identity may be used. For example, in the case of an airport, a boarding pass may be used. The user's identity and the corresponding identification such as the identity card number or passport number may be known by scanning the barcode on the boarding pass and querying an airline's database. For another example, in the case of a station, if a real-name ticket system is adopted, the user's identity and the corresponding identification may be known by scanning a two-dimensional code on the ticket and querying a ticketing database of the station.", + "p0047": "Certainly, the identity information entry device 107 is not limited to entering the identification of the inspected person, but may also input other identity information. For example, as described in the following embodiments, the identity information entry device 107 may also acquire a registered photo of the inspected person. Here, the \u201cregistered photo\u201d refers to a photo used by the inspected person to clearly reflect his or her appearance when registering his or her identity with a trusted institution. For example, such a registered photo may be a photo used by the inspected person to apply for the identity card or passport in a police office.", + "p0048": "The identity information entry device 107 may obtain the above registered photo directly from the identity document with photo printed thereon , or may obtain the above registered photo from a relevant trusted database based on the entered identification.", + "p0049": "The identity information entry device 107 may be placed at the entrance of the security inspection lane 101 . When the inspected person enters the security inspection lane 101 , his or her identification may be entered by the identity information entry device 107 and transmitted to the control device 105 via a wired or wireless link. The control device 105 may obtain user data related to the inspected person identified by the identification from databases 111 - 1 , 111 - 2 , . . . , 111 - n via a network 109 . Here, the user data may include various data capable of characterizing the user, such as data related to the identity and behavior of the inspected person, including but not limited to one or more of personal data, credit, social relationship, or historical behavior of the inspected person. Here, the user data may reflect sociological characteristics of the user, rather than biological features .", + "p0050": "With the rapid development of the Internet, the Internet of Things and the mobile Internet, various types of data have shown significant growth in recent years, and studies have shown that the amount of data will increase exponentially in the next few years. In the era of big data, user data are widely distributed in various fields. FIG. 2 is a schematic diagram showing various user data. For example, basic personal information of the user, including information such as name, gender, age, former name, place of birth, contact information, employer, residence address, marriage history, and kinship, may be obtained through the personal information inquiry system of the Ministry of Public Security. Further, call history, communication records, and Internet records of the user may be obtained through telecommunication operators. By analyzing the data, it is possible to obtain contact information, duration of each call, frequency of making calls, time and location of each visit to the network, and IP addresses of visited web sites, or the like for the user. Furthermore, bank card information, transaction information on clothing, food, housing, transportation or the like, credit status, and debt information of the user may be obtained through banking systems. Furthermore, personal consumption details, categories of browsed products, payment habits, online transaction details, and user contact information of the user may be obtained through e-commerce and B2C enterprises. Furthermore, all traces left by the user on the Internet, such as websites that the user frequently visits and their themes and contents, log information of visits, website browsing records, and chat information, shared information and comment information in social networking sites such as WeChat and Weibo, or the like may be tracked through the Internet.", + "p0051": "The user data is of significant value, and there is some connection, more or less, between pieces of data. Through the analysis of the above data, comprehensive information of the user, such as the user's hobbies, personality characteristics, thought fluctuations, emotional changes, family relationships, personal connections, work experience, economic status, credit status, debt situation, living habits, behavior habits, consumption habits and travel tracks, may be inferred. The potential hazard from the user may be predicted from the data such as the user's family relationship, employment status, interpersonal relationship, whether or not there has been a significant change in his or her family or around, or the like. The user's recent consumption records may be obtained from the user's consumption data, and by analyzing the consumption records, it is possible to obtain information such as whether the user has purchased contraband goods, dangerous goods, and materials that may be used to make dangerous goods. In addition, by analyzing the consistency of the user's recent consumption records and his or her past consumption habits, it is possible to predict the possibility that the user will take abnormal behaviors that might be harmful to the society. Further, it is possible to know the possibility that the user is to make anti-social words and actions through information on web pages frequently browsed by the user, messages forwarded by the user, and speeches made by the user. Furthermore, it is possible to indirectly predict the security level of the user according to security levels of the user's common contacts by drawing a personal relationship map for the user which is centered at the user from the user's activity data in the social networks, and comprehensively analyzing data such as social activities of other users who interact with the user frequently.", + "p0052": "According to an embodiment of the present disclosure, the system 100 may include a security factor determination device for acquiring the user data related to the inspected person based on the identification of the inspected person, and determining the security factor of the inspected person based on the acquired user data. In this example, the security factor determination device may be implemented, for example, by running program instructions by the control device 105 . However, the present disclosure is not limited thereto. Alternatively, the security factor determination device may be implemented by a separate entity, such as a computing device. For example, the control device 105 may extract data related to the identity and behavior of different persons from a large amount of data by, for example, the data mining technology, and classify those persons based thereon, and apply different levels of security inspection policies to different categories of persons.", + "p0053": "For example, a scoring mechanism may be established to characterize the security level of the inspected person based on his or her security factor. The higher is the security factor, the more normal is the inspected person. The corresponding security factor of the inspected person may be used as a guide for setting the parameters used in the security inspection device to perform the security inspection on the inspected person , thereby realizing differentiated and personalized security inspection for different inspected persons. In reality, some inspected persons' data may be missing or only partially available. Such inspected persons may be assigned lower security factors so that they will be subject to strengthened inspection, in order to prevent adverse effects due to false negatives during the security inspection process. In the following, the determination of the security factor will be described in further detail.", + "p0054": "In addition, the user data may further include historical security inspection data, including historical departure and destination locations, and results of previous security inspections themselves and also their baggage, or the like). In establishing the user security factor model, pieces of data including some special locations and bad historical records may have increased weights assigned thereto, that is, these pieces of data may have increased impacts on the safety factor.", + "p0055": "According to an embodiment of the present disclosure, the system 100 may include a parameter determination device for determining a parameter for performing the security inspection on the inspected person based on the determined security factor of the inspected person. In this example, the parameter determination device may be implemented, for example, by the control device 105 running program instructions. However, the present disclosure is not limited thereto. The parameter determination device may also be implemented by a separate entity, such as a computing device. The basic principle of the parameter determination is as follows: for inspected persons with relatively high security factor values, relatively loose parameters may be set to appropriately reduce the intensity of the security inspection on those persons, and thus to improve the experience of the security inspection device by those relatively safe persons and speed up the passage of those persons;", + "p0056": "while for persons with relatively low security factor values, relatively strict parameters may be set to enhance the intensity of the security inspection on those persons.", + "p0057": "These parameters may be any parameters in the system related to the security inspection intensity, such as parameters related to hardware and/or parameters related to software . Certainly, it is less convenient to constantly change the hardware parameters for different inspected persons, so it is preferable to change the software parameters. For example, the parameters may be parameters applied in the hidden object recognition algorithm employed by the security inspection device, including one or more of a classifier type, a classifier parameter, or an alarm threshold. For example, a SVM classifier may be used. In this case, the classifier type may be controlled by adjusting a kernel function. The classifier parameter may include a penalty factor or the like. The alarm threshold is a distance between a classification plane corresponding to the classifier and an optimal segmentation hyperplane. The closer the classification plane is to the category of suspected goods, the smaller is the alarm threshold. For the inspected persons with higher security factors, it is possible to use a linear classifier with higher generalization ability or a SVM classifier corresponding to a smaller penalty factor, and to increase the alarm threshold; while for the inspected persons with lower security factors, it is possible to use a nonlinear classifier or a SVM classifier corresponding to a penalty factor, which may be increased to increase the positive detection rate, and to reduce the alarm threshold, so as to avoid losses due to the false negatives.", + "p0058": "According to an embodiment of the present disclosure, the system 100 may further include a video device 113 such as a camera for capturing an image of the inspected person in real time. In this example, the video device 113 is illustrated as being disposed on the scanning device 103 , but the disclosure is not limited thereto. The video device 113 may be placed at any other location in the security inspection lane. The number of the video device 113 is not limited to one, but may be more. The video device 113 may be arranged such that its or their capturing or monitoring range may cover the security inspection lane 101 so that the inspected person can be monitored in real time without dead space when the inspected person is passing through the security inspection lane 101 . As mentioned above, the security inspection lane may also be defined by the monitoring range of the video device 113 .", + "p0059": "In the case where the video device 113 is provided, the system 100 may further include an abnormal behavior determination device for determining whether the inspected person has an abnormal behavior based on the image of the inspected person taken by the video device 113 . In this example, the abnormal behavior determination device may be implemented, for example, by the control device 105 running program instructions. In this case, the video device 113 may be connected to the control device 105 by wire or wirelessly. However, the present disclosure is not limited thereto. The abnormal behavior determination device may also be implemented by a separate entity such as a computing device. The control device 105 may use the computer vision technology to extract a contour image of the inspected person from the video collected by the video device 113 and perform dynamic tracking, and use a person modeling model pre-trained by the machine learning method to monitor the abnormal behavior. Once the inspected person is found to have abnormal behavior, the policemen may be notified in a timely manner.", + "p0060": "In addition, the system 100 may further include an authentication device for extracting a face image of the inspected person from the image taken by the video device 113 , and verifying the identity of the inspected person by comparing the extracted face image with the registered photo of the inspected person. In this example, the authentication device may be implemented, for example, by the control device 105 running program instructions. However, the present disclosure is not limited thereto. The authentication device may also be implemented by a separate entity, such as a computing device. Here, the registered photo of the inspected person is, for example, a registered photo input by the identity information entry device 107 , or a registered photo obtained from the relevant trusted databases as described above. If there is some inconsistency between the person and the identity document, the system may issue an alarm to the security personnel to carry out manual verification on the inspected person, and also open the baggage carried by the inspected person for more careful inspection.", + "p0061": "On the other hand, the authentication device may also compare the extracted face image with face images from a database of untrusted persons. For example, the database of untrusted persons may include a face image database of fugitive suspects provided by the Ministry of Public Security, a face image database of persons restricted from entering or leaving the country or restricted from leaving a certain place published by the court, or the like. If there is a high degree of matching, relevant personnel may be notified to conduct a security review.", + "p0062": "The control device 105 may configure the security inspection device using the parameters determined by the parameter determination device by, for example, configuring parameters applied in the hidden object recognition algorithm employed by the security inspection device . By processing the image of the inspected person obtained by the scanning device 103 with the algorithm thus configured, it is possible to identify whether or not the inspected person has concealed some article. The inspection results may be output to the security personnel.", + "p0063": "According to an embodiment of the present disclosure, complementary displays at a remote end and the device end may be employed. Specifically, at the security inspection device , only a doll picture reflecting the contour of the human body and corresponding alarm information and identity information of the inspected person may be displayed. On the other hand, a remote console may be provided in a closed monitoring room at a certain distance from the security inspection device. The actual scanned image is displayed only on the remote console, with the face blurred. The security inspector at the remote console may view and process the scanned image of the current inspected person, but does not know any personal information related to the identity of the current inspected person. At the remote console, the scanned image of the inspected person may be deleted immediately upon completing a scan and corresponding inspection. Further, the doll picture displayed on the security inspection device does not have any surface information of the scanned person, thereby enabling adequate protection of the personal privacy of the inspected person.", + "p0064": "That is to say, on the remote end and the device end, the security inspection results may be displayed in a manner that complements each other, without an intersection therebetween. In this way, it is possible to, on one hand, protect the personal privacy of the inspected person, and on the other hand, clearly review the inspection result of the inspected person.", + "p0065": "In the above embodiments, the centralized control device 105 is shown, but the present disclosure is not limited thereto. Distributed control devices are also possible. For example, one or more components of the system 100 may have a corresponding control device, while other components may have other control device .", + "p0066": "In the above examples, the security factor determination device is incorporated in the security inspection system 100 . However, the present disclosure is not limited thereto. For example, multiple security inspection systems may share the same security factor determination device. For example, at an airport, security inspection systems deployed at different security inspection lanes may share a server that acts as a security factor determination device. Alternatively, the security factor determination device may be provided by a professional data analysis company. The security inspection system 100 may send the entered identification of the inspected person to a server of the data analysis company, and the server of the data analysis company retrieves relevant user information and performs data analysis to obtain the security factor of the inspected person. Subsequently, the security factor may be sent to the security inspection system 100 , and then the parameter determination device may determine the security inspection parameters for the inspected person based on the received security factor.", + "p0067": "Hereinafter, an operational flow of the above-described security inspection system will be described with reference to FIG. 3 .", + "p0068": "First, as shown at 310 , model training may be performed to establish a relationship model between user data and security factors. This may be done in the security inspection system 100 described above, or may be performed outside of the security inspection system 100 by, for example, the professional data analysis company described above.", + "p0069": "Specifically, as shown at 311 , a plurality of databases may be used to respectively receive a large amount of shared data from third-party organizations that have established partnerships, including name data, age data, place of birth data, residence data, document number data, interpersonal relationship data, communication data, transaction data, credit data, social activity data, social network data, and other user-related data. These massive data are cleaned and pre-processed and imported into a large distributed database. The user data may be stored hierarchically according to key information such as age, region and occupation, for convenience of reading the data later. For example, data such as texts, images, audio, video, or the like related to users may be obtained from departments with rich and reliable source data such as the Ministry of Public Security, operators, banks, e-commerce, B2C companies, and Internet companies, and these massive data may be formatted and imported into a large distributed database for storage.", + "p0070": "Then, as shown at 313 , the massive data embodying user information may be analyzed and processed by methods such as data mining, machine learning and statistical analysis to establish the relationship model between user data and security factors. According to an embodiment of the present disclosure, the user data may be clustered into several categories, and each category is assigned a different security factor value. For example, such a model may be built as follows.", + "p0071": "Suppose that the large amount of user data extracted from the database are S= , where X n = , N represents the total number of pieces of the user data, X n represents the data of the n-th user, |X n | represents the dimension of the data of the n-th user, and x j represents data information of the j-th dimension of the n-th user, which may be structured data such as numbers, or may be unstructured data such as text, image, audio, video, or web page. Referring to FIG. 4 , these user data may be represented as points in a data space.", + "p0072": "The user data may be clustered into several categories. For example, the K-means clustering algorithm may be used to classify the input user data into K categories according to their respective characteristics, with centers of the respective categories represented by C= . The classification result is schematically shown by dotted circles in FIG. 4 . According to an embodiment of the present disclosure, each of the categories is assigned a different security factor value according to the security level presented by the users corresponding to that category. As shown in FIG. 4 , the 0th category has the highest security level, the 1th category has the second highest security level, . . . , and the -th category has the lowest security level. Here, the category center C i is used to indicate the i-th security level. That is, users of the same category have the same security level . For C i , the larger i is, the lower the security level of the users of the corresponding category.", + "p0073": "Then, the relationship model between user data and security factors is established. For example, the relationship between user data x and its security factor may be expressed by D=f , where", + "p0075": "where k=0, . . . , K\u22121, wherein", + "p0077": "x belongs to the k-th category , and ||\u22c5|| represents Euclidean distance.", + "p0078": "Different users may have different security factors due to the difference between their respective data, and the security factor may range from that for the data belonging to the same category, the farther the user data is from the center of the category, the larger the corresponding security factor D is. However, this is just an example. Other models may be created in which, for example, for the same category of data, substantially the same security factor may be assigned.", + "p0079": "The established model may be stored at the security factor determination device , or may be stored outside of the security factor determination device .", + "p0080": "In the actual security inspection process 320 , the identification of the user is first entered by the identity information entry device 107 . In the case of entering the registered photo, authentication may be performed at 321 . For example, as described above, authentication may be performed by comparing the registered photo with the face image from the image captured by the video device 113 .", + "p0081": "The security factor determination module may obtain user data based on the entered user identification and determine the security factor for the user using the model established as described above, as indicated at 323 . Certainly, the control device 105 may also send the entered user identification to a security factor determination module external to the system and receive the determined security factor from the external security factor determination module. After the user's security factor is determined, the security inspection device may be configured accordingly. For example, as indicated at 325 , parameters of the security inspection device, such as parameters applied in the hidden object recognition algorithm, including the classifier type, the classifier parameter, the alarm threshold, or the like, may be generated based on the security factor by, for example, a self-learning algorithm. These parameters may be represented in the form of a parameter vector.", + "p0082": "Specifically, the relationship between the current user data x and the K category centers may be calculated to determine the category center C k of the user, and then the user data x may be substituted into the above model D=f , thereby calculating the security factor of the current inspected person. The security level C k corresponding to the user may also be output to the device end and the remote end to notify the security personnel of the security level of the user, so as to prompt the security personnel to be ready to take appropriate measures. As a result, it is possible to improve the throughput of the device and reduce the occurrence of dangerous events.", + "p0083": "Through the analysis of the user's security factor, the inspection measures and inspection strength may be adjusted accordingly. As mentioned above, for users with a large security factor value , the intensity of the security inspection on those persons may be appropriately reduced to improve the experience of the security inspection device by such relatively safe persons and speed up the passage of those persons; while for users with a small security factor value , the intensity of the security inspection of such users may be enhanced, and the relevant information such as the security level of such users may be provided by the system to the security staff for early warning, prompting the staff to take corresponding protective measures.", + "p0084": "The security inspection device may receive the security inspection parameters, for example, sent in the form of a parameter vector, configure itself with the received parameters, and perform security inspection on the user. For example, millimeter waves may be emitted to illuminate the user to image the user based on millimeter waves from the user, and the image may be processed using a hidden object recognition algorithm, as shown at 327 . The results of the inspection may be displayed as shown at 329 . As mentioned above, the display may be done in a complementary manner at the remote end and the device end.", + "p0085": "According to an embodiment of the present disclosure, the data may also be updated as indicated at 330 .", + "p0086": "At the end of each security inspection, the data of the current inspected person may be updated. For example, the information involved in the security inspection process may be collected in real time and stored in the corresponding database, as shown at 331 . The data update consists of two parts, namely, the information of the present security inspection process and the personal information of the inspected person. For example, the security inspection process information includes the number and location of the current security inspection device, information of the on-duty security inspector, or the like. The personal information of the inspected person includes the document information, the travel location, the face image, the video data recorded in the security inspection lane, the imaging data, the results of the present security inspection, or the like.", + "p0087": "In addition, in the era of big data, user data are being constantly updated, and the users' security factors are constantly changing as the data changes. Therefore, as shown at 333 , the large-scale data provided by the third-party platform may be grasped and updated in real time to ensure the real-time and reliability of the security inspection.", + "p0088": "According to embodiments of the present disclosure, it is possible to accurately predict the user's behavior and evaluate the risk or potential danger from the user by analyzing and mining the user's comprehensive data, and thus to provide a more accurate security inspection solution.", + "p0089": "The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the respective embodiments are separately described above, this does not mean that the measures in the respective embodiments cannot be used in combination to advantage. The scope of the disclosure is defined by the appended claims and their equivalents. Numerous alternatives and modifications may be made by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications should all fall within the scope of the present disclosure.", + "features": { + "F1": "The security inspection system comprises an identity information entry device configured to enter an identification of an inspected person.", + "F2": "The security inspection system includes a security factor determination device configured to grasp user data of the inspected person from a third-party platform in real-time and determine the security factor by substituting user data into a relationship model between user data and security factors.", + "F3": "A parameter determination device is configured to determine parameters for performing security inspections based on the security factor of the inspected person, including one or more of a classifier type, classifier parameter, and alarm threshold in a trained hidden object recognition algorithm.", + "F4": "The security inspection device is configured to perform the security inspection on the inspected person by processing a scanning image and using the determined parameter to identify if an article is concealed.", + "F5": "The security inspection device extracts features of the inspected person, calculates projected vectors of features in a dictionary, and inputs the projected vectors into a classifier corresponding to the determined parameter for classification.", + "F6": "User data of the inspected person is updated at the end of each security inspection which includes real-time updates to the user data on the third-party platform, and the relationship model is established using the updated data for offline model training.", + "F7": "The relationship model classifies user data from the third-party platform into categories with different security factors determined by Euclidean distance between user data of the inspected person and a center of each category.", + "F8": "The security factor comprises two levels: a first security factor and a second security factor less than the first, with associated parameters for inspection involving different types of classifiers and penalty factors.", + "F9": "The identity information entry device may also obtain a registered photo of the inspected person.", + "F10": "An authentication device can extract a face image from videos, verifying the identity by comparing it with registered photos or a database of untrusted persons.", + "F11": "A video device is configured to capture real-time images of the inspected person.", + "F12": "An abnormal behavior determination device is configured to detect abnormal behavior based on images captured by the video device.", + "F13": "User data can include personal data, credit, social relationships, or historical behavior.", + "F14": "A display device is configured to show the corresponding security level of the inspected person during the inspection process." + } + } + }, + { + "FAN": "76109734", + "CPC": [ + "G06F-011/07", + "G06F-011/07/09", + "G06F-011/07/51", + "G06F-011/07/66", + "G06F-011/07/75", + "G06F-011/07/87", + "G06F-011/07/9", + "G06F-011/07/93", + "G06F-016/00", + "G06F-016/958", + "G06F-017/30/89" + ], + "Content": { + "title": "Identifying potential solutions for abnormal events based on historical data", + "pa01": "Techniques for identifying potential solutions for abnormal events based on historical data are described. One example method includes obtaining interaction data between a client device and a server; determining whether an abnormal event exists in the obtained interaction data; in response to determining, obtaining attribute information of a user corresponding to the client device from the obtained interaction data; identifying one or more records in a database, each record including the attribute information, a cause of one or more previous abnormal events, and a weight characterizing a severity of the previous abnormal events; determining a cause of the abnormal event, wherein the determined cause is the same as the cause of the previous abnormal events from the record in the identified one or more records that includes a largest weight; and providing a solution to address the abnormal event based on the determined cause.", + "c-en-0001": "A computer-implemented method performed by one or more processors, comprising: obtaining, by the one or more processors, interaction data between a client device and a server; determining, by the one or more processors, whether an abnormal event exists in the obtained interaction data; in response to determining that the abnormal event exists in the obtained interaction data, obtaining, by the one or more processors, attribute information of a user corresponding to the client device from the obtained interaction data; identifying, by the one or more processors, one or more records in a database, each record including the attribute information, a cause of one or more previous abnormal events, and a weight characterizing a severity of the previous abnormal events; determining, by the one or more processors, a cause of the abnormal event, wherein the determined cause is the same as the cause of the previous abnormal events from the record in the identified one or more records that includes a largest weight; and providing, by the one or more processors, a solution to address the abnormal event based on the determined cause.", + "c-en-0002": "The computer-implemented method of claim 1 , further comprising: determining, by the one or more processors, an occurrence of a plurality of abnormal events in the obtained interaction data; sorting, by the one or more processors, the plurality of abnormal events in a predetermined abnormal event order; and executing, by the one or more processors, in a sequential order of the sorted plurality of abnormal events, the sorted plurality of abnormal events based on causes for each abnormal event in the sorted plurality of abnormal events.", + "c-en-0003": "The computer-implemented method of claim 1 , wherein the weight characterizing the severity of the abnormal event includes a quantity of times the abnormal event occurred.", + "c-en-0004": "The computer-implemented method of claim 1 , further comprising: executing, by the one or more processors, the solution to address abnormal event to produce a processing result.", + "c-en-0005": "The computer-implemented method of claim 4 , further comprising: receiving, by the one or more processors, feedback data from a user of the client device based on the processing result obtained from the server after processing the abnormal event.", + "c-en-0006": "The computer-implemented method of claim 5 , wherein the feedback data comprises an indication of an agreement on the executed solution or an indication of a disagreement on the executed solution, as indicated by the user at the client device.", + "c-en-0007": "The computer-implemented method of claim 1 , wherein identifying the one or more records in the database further comprises: comparing, by the one or more processors, attribute information from each of the one or more records in the database to the attribute information from the obtained interaction data; and in response to comparing the attribute information from each of the one or more records in the database to the attribute information found in the obtained interaction data, determining, by the one or more processors, a match between the attribute information from at least one of the one or more records to the attribute information from the obtained interaction data.", + "c-en-0008": "A non-transitory, computer-readable medium storing one or more instructions executable by a computer system to perform operations comprising: obtaining, by one or more processors, interaction data between a client device and a server; determining, by the one or more processors, whether an abnormal event exists in the obtained interaction data; in response to determining that the abnormal event exists in the obtained interaction data, obtaining, by the one or more processors, attribute information of a user corresponding to the client device from the obtained interaction data; identifying, by the one or more processors, one or more records in a database, each record including the attribute information, a cause of one or more previous abnormal events, and a weight characterizing a severity of the previous abnormal events; determining, by the one or more processors, a cause of the abnormal event, wherein the determined cause is the same as the cause of the previous abnormal events from the record in the identified one or more records that includes a largest weight; and providing, by the one or more processors, a solution to address the abnormal event based on the determined cause.", + "c-en-0009": "The non-transitory, computer-readable medium of claim 8 , the operations further comprising: determining, by the one or more processors, an occurrence of a plurality of abnormal events in the obtained interaction data; sorting, by the one or more processors, the plurality of abnormal events in a predetermined abnormal event order; and executing, by the one or more processors, in a sequential order of the sorted plurality of abnormal events, the sorted plurality of abnormal events based on causes for each abnormal event in the sorted plurality of abnormal events.", + "c-en-0010": "The non-transitory, computer-readable medium of claim 8 , wherein the weight characterizing the severity of the abnormal event includes a quantity of times the abnormal event occurred.", + "c-en-0011": "The non-transitory, computer-readable medium of claim 8 , the operations further comprising: executing, by the one or more processors, the solution to address abnormal event to produce a processing result.", + "c-en-0012": "The non-transitory, computer-readable medium of claim 11 , the operations further comprising: receiving, by the one or more processors, feedback data from a user of the client device based on the processing result obtained from the server after processing the abnormal event.", + "c-en-0013": "The non-transitory, computer-readable medium of claim 12 , wherein the feedback data comprises an indication of an agreement on the executed solution or an indication of a disagreement on the executed solution, as indicated by the user at the client device.", + "c-en-0014": "The non-transitory, computer-readable medium of claim 8 , wherein identifying the one or more records in the database further comprises: comparing, by the one or more processors, attribute information from each of the one or more records in the database to the attribute information from the obtained interaction data; and in response to comparing the attribute information from each of the one or more records in the database to the attribute information found in the obtained interaction data, determining, by the one or more processors, a match between the attribute information from at least one of the one or more records to the attribute information from the obtained interaction data.", + "c-en-0015": "A computer-implemented system, comprising: one or more computers; and one or more computer memory devices interoperably coupled with the one or more computers and having tangible, non-transitory, machine-readable media storing one or more instructions that, when executed by the one or more computers, perform one or more operations comprising: obtaining, by one or more processors, interaction data between a client device and a server; determining, by the one or more processors, whether an abnormal event exists in the obtained interaction data; in response to determining that the abnormal event exists in the obtained interaction data, obtaining, by the one or more processors, attribute information of a user corresponding to the client device from the obtained interaction data; identifying, by the one or more processors, one or more records in a database, each record including the attribute information, a cause of one or more previous abnormal events, and a weight characterizing a severity of the previous abnormal events; determining, by the one or more processors, a cause of the abnormal event, wherein the determined cause is the same as the cause of the previous abnormal events from the record in the identified one or more records that includes a largest weight; and providing, by the one or more processors, a solution to address the abnormal event based on the determined cause.", + "c-en-0016": "The computer-implemented system of claim 15 , the operations further comprising: determining, by the one or more processors, an occurrence of a plurality of abnormal events in the obtained interaction data; sorting, by the one or more processors, the plurality of abnormal events in a predetermined abnormal event order; and executing, by the one or more processors, in a sequential order of the sorted plurality of abnormal events, the sorted plurality of abnormal events based on causes for each abnormal event in the sorted plurality of abnormal events.", + "c-en-0017": "The computer-implemented system of claim 15 , wherein the weight characterizing the severity of the abnormal event includes a quantity of times the abnormal event occurred.", + "c-en-0018": "The computer-implemented system of claim 15 , the operations further comprising: executing, by the one or more processors, the solution to address abnormal event to produce a processing result.", + "c-en-0019": "The computer-implemented system of claim 18 , the operations further comprising: receiving, by the one or more processors, feedback data from a user of the client device based on the processing result obtained from the server after processing the abnormal event.", + "c-en-0020": "The computer-implemented system of claim 19 , wherein the feedback data comprises an indication of an agreement on the executed solution or an indication of a disagreement on the executed solution, as indicated by the user at the client device.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATION", + "p0002": "This application is a continuation of PCT Application No. PCT/CN2016/104184, filed on Nov. 1, 2016, which claims priority to Chinese Patent Application No. 201510763290.X, filed on Nov. 10, 2015, and each application is hereby incorporated by reference in its entirety.", + "p0003": "TECHNICAL FIELD", + "p0004": "The present invention relates to the field of computer technologies, and more particularly, to techniques for identifying potential solutions for abnormal events based on historical data.", + "p0005": "BACKGROUND", + "p0006": "Currently, a user often performs operations such as browsing a web page, making an online payment, or registering an account on a terminal device. However, sometimes these operations cannot be successfully performed for various reasons. For example, online payment can fail or a web page browsed cannot be displayed.", + "p0007": "Generally, when encountering a problem, a user usually calls a customer service representative for help. After the call is connected, the user needs to describe the problem to the customer service representative, such as a symptom related to the problem, latest operations performed by the user, or information returned by a server. The customer service representative determines a cause of the problem based on description from the user and expertise of the customer service representative, and then helps the user resolve the problem.", + "p0008": "However, the user sometimes cannot clearly describe the problem, or the customer service representative cannot thoroughly understand the description from the user right away because of different perspectives of the problem. As a result, it can take additional communication time for the customer service representative to determine the cause of the problem, thereby increasing labor costs.", + "p0009": "SUMMARY", + "p0010": "To overcome problems in related technologies, the present invention provides a data processing method and apparatus.", + "p0011": "According to a first aspect of implementations of the present invention, a data processing method is provided and applied to a server, where the method includes: obtaining interaction data between a user and the server; detecting, based on the interaction data, whether an abnormal event has occurred; obtaining attribute information of the user when an abnormal event occurred; searching a locally stored first correspondence between attribute information corresponding to the abnormal event, a cause, and a weight for a record that includes the attribute information and a largest weight; and determining a cause in the record as a cause of the abnormal event.", + "p0012": "The method further includes: processing the abnormal event based on the determined cause. When a plurality of abnormal events have occurred, the processing of the abnormal event based on the determined cause includes: sorting the plurality of abnormal events in a predetermined abnormal event order; and sequentially processing, in an order of the sorted abnormal events, the abnormal events based on causes of the abnormal events until the abnormal events are processed.", + "p0013": "The processing of the abnormal event based on the determined cause includes: obtaining a locally stored second correspondence between a cause corresponding to the abnormal event and a processing solution; obtaining, from the second correspondence, a processing solution corresponding to the determined cause; and processing the abnormal event based on the processing solution.", + "p0014": "After the processing of the abnormal event based on the determined cause, the method further includes: receiving feedback data that is fed back by the user based on a processing result obtained from the server that has processed the abnormal event; and when the feedback data includes an agreement identifier, adding a weight in a record that includes the attribute information and the determined cause to the first correspondence; or when the feedback data includes a disagreement identifier, receiving a cause of the abnormal event that is sent by a customer service representative, where the received cause is sent after the customer service representative interacts with the user; and if the first correspondence includes a record that includes the attribute information and the received cause, adding a weight in the record that includes the attribute information and the received cause; or if the first correspondence does not include a record that includes the attribute information and the received cause, setting an initial weight, and storing the attribute information, the received cause, and the initial weight in the first correspondence as a single record.", + "p0015": "Based on a second aspect of the implementations of the present invention, a data processing apparatus is provided and applied to a server, where the apparatus includes: a first acquisition module, configured to obtain interaction data between a user and the server; a detection module, configured to detect, based on the interaction data, whether an abnormal event occurred; a second acquisition module, configured to obtain attribute information of the user when an abnormal event occurred; a search module, configured to search a locally stored first correspondence between attribute information corresponding to the abnormal event, a cause, and a weight for a record that includes the attribute information and a largest weight; and a determining module, configured to determine a cause in the record as a cause of the abnormal event.", + "p0016": "The apparatus further includes: a processing module, configured to process the abnormal event based on the determined cause. When a plurality of abnormal events have occurred, and the processing module includes: a sorting unit, configured to sort the plurality of abnormal events in a predetermined abnormal event order; and a first processing unit, configured to sequentially process, in an order of the sorted abnormal events, the abnormal events based on causes of the abnormal events until the abnormal events are processed.", + "p0017": "The processing module includes: a first acquisition unit, configured to obtain a locally stored second correspondence between a cause corresponding to the abnormal event and a processing solution; a second acquisition unit, configured to obtain, from the second correspondence, a processing solution corresponding to the determined cause; and a second processing unit, configured to process the abnormal event based on the processing solution.", + "p0018": "The apparatus further includes: a first receiving module, configured to receive feedback data that is fed back by the user based on a processing result obtained from the server that has processed the abnormal event; and a first addition module, configured to: add a weight in a record that includes the attribute information and the determined cause to the first correspondence, when the feedback data includes an agreement identifier; or a second receiving module, configured to: receive a cause of the abnormal event that is sent by a customer service representative, when the feedback data includes a disagreement identifier, where the received cause is sent after the customer service representative interacts with the user; and a second addition module, configured to: add a weight in the record that includes the attribute information and the received cause, if the first correspondence includes a record that includes the attribute information and the received cause; or a storage module, configured to: set an initial weight, store the attribute information, the received cause, and the initial weight in the first correspondence as a single record, if the first correspondence does not include a record that includes the attribute information and the received cause.", + "p0019": "The technical solutions provided in the implementations of the present invention can deliver the following beneficial effects: In the present invention, the interaction data between the user and the server is obtained; whether an abnormal event occurred is detected based on the interaction data; when an abnormal event occurred, the attribute information of the user is obtained; and the locally stored first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight is searched to identify the record that includes the attribute information and the largest weight; and the cause in the record is determined as the cause of the abnormal event. In this process, the user does not need to communicate with the customer service representative, the customer service representative does not need to determine the cause of the abnormal event by communicating with the user, and the cause of the abnormal event can be determined by using the attribute information of the user and the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight. Therefore, labor costs are saved.", + "p0020": "It should be understood that the previous general description and the following detailed description are only examples and are explanatory. They do not limit the present invention.", + "p0021": "BRIEF DESCRIPTION OF DRAWINGS", + "p0022": "The accompanying drawings herein are incorporated in the specification, constitute a part of the specification, show implementations that are in accordance with the present invention, and are used with the specification to explain principles of the present invention.", + "p0023": "FIG. 1 is a flowchart illustrating a data processing method based on an example implementation;", + "p0024": "FIG. 2 is a schematic diagram illustrating a decision tree model based on an example implementation;", + "p0025": "FIG. 3 is a flowchart illustrating a data processing method based on an example implementation;", + "p0026": "FIG. 4 is a flowchart illustrating a data processing method based on an example implementation;", + "p0027": "FIG. 5 is a block diagram illustrating a data processing apparatus based on an example implementation; and", + "p0028": "FIG. 6 is a flowchart illustrating an example of a computer-implemented method for identifying potential solutions for abnormal events based on historical data, according to an implementation of the present disclosure.", + "p0029": "DESCRIPTION OF EMBODIMENTS", + "p0030": "Example implementations are described in detail here, and examples of the implementations are presented in the accompanying drawings. When the following description relates to the accompanying drawings, unless specified otherwise, same numbers in different accompanying drawings represent same or similar elements. Implementations described in the following implementations do not represent implementations based on the present invention. Instead, they are only examples of apparatus and methods based on some aspects of the present invention that are described in detail in the appended claims.", + "p0031": "FIG. 1 is a flowchart illustrating a data processing method based on an example implementation. As shown in FIG. 1 , the method is applied to a server, and the method includes the following steps.", + "p0032": "In step S 101 , interaction data between a user and the server is obtained.", + "p0033": "The interaction data includes data sent by the user to the server and data returned by the server to the user.", + "p0034": "For example, when the user needs to obtain a web page, the user can send to the server, an acquisition request used to obtain the web page, where the acquisition request includes a link of the web page. After receiving the acquisition request, the server obtains the web page based on the link, and can return an acquisition response to the user based on the acquisition request. The acquisition request and the acquisition response are interaction data between the user and the server.", + "p0035": "As another example, when needing to pay for an item online, the user sends a payment request to the server, where the payment request includes a user account, cost of the item, and a seller account. After receiving the payment request, the server processes the payment request, for example, deducting the cost of the item from a balance included in the user account, then adding the deducted cost of the item to a balance of the seller account, and then returns a payment response to the user. The payment request and the payment response are interaction data between the user and the server.", + "p0036": "In step S 102 , whether an abnormal event occurred is detected based on the interaction data.", + "p0037": "For example, when a format of a link included in the acquisition request sent by the user to the server and used to obtain the web page is incorrect, the server cannot obtain the web page based on the link. It returns an acquisition response to the user to notify that the web page is not obtained. The acquisition response is interaction information between the user and the server, and the acquisition response indicates that the web page is not successfully obtained. Therefore, it can be determined that an abnormal event occurred.", + "p0038": "As another example, after receiving the payment request, the server deducts the cost of the item from the balance included in the user account. However, if the balance included in the user account is less than the price of the item, the cost of the item cannot be successfully deducted from the balance of the user account, and the cost of the item cannot be added to the balance included in the seller account. As a result, the processing request fails to be processed, and a payment response used to notify that the acquisition request fails to be processed is returned to the user. The payment response is interaction information between the user and the server, and the payment response indicates that payment for the item is unsuccessful. Therefore, it can be determined that an abnormal event occurred.", + "p0039": "When an abnormal event occurs, in step S 103 , attribute information of the user is obtained.", + "p0040": "The attribute information of the user can be gender, age, etc. of the user stored in the user account, whether the user account is bound to a bank card, whether real-name authentication is performed for the user account, whether a payment function of the user account is enabled, etc.", + "p0041": "In step S 104 , a locally stored first correspondence between attribute information corresponding to the abnormal event, a cause, and a weight is searched to identify a record that includes the attribute information and the largest weight.", + "p0042": "The first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight can be obtained in advance by a technician through configuration, or can be obtained by the server through auto-learning.", + "p0043": "The server can obtain, through auto-learning, the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight using the following process:", + "p0044": "In a historical process, when an abnormal event occurred during interaction between a user and the server, a customer service representative can communicate with the user. During communication, the customer service representative determines attribute information of the user and a cause of the abnormal event based on information about the abnormal event described by the user, and then enters the determined attribute information of the user and the determined cause provided to the server. The server receives the attribute information and the cause entered by the customer service representative.", + "p0045": "The locally stored first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight are searched to identify a record that includes the attribute information and the cause entered by the customer service representative. When a record includes the attribute information and the cause entered by the customer service representative, a weight in the record that includes the attribute information and the cause entered by the customer service representative is added. When no record includes the attribute information and the cause entered by the customer service representative, an initial weight is set, and the attribute information and the cause entered by the customer service representative, and the initial weight are stored in the first correspondence as a single record.", + "p0046": "Any records in the first correspondence include attribute information, a cause, and a weight. In a historical process, when the attribute information of the user is the attribute information in the record, and the cause is the cause in the record, the weight can be a quantity of times the abnormal event occurred. This holds true for other records in the first correspondence.", + "p0047": "Therefore, in this step, the first correspondence can be searched to identify records that include the attribute information. When a record includes the attribute information, the record that includes the largest weight is selected from the records that includes the attribute information, and step S 105 is performed.", + "p0048": "However, if no record includes the attribute information, to accurately determine the cause of the abnormal event, the server can associate a connection between the server and the user's terminal device with a connection between the server and the customer service representative's terminal device, so that the terminal device of the user is directly connected to the terminal device of the customer service representative, and the customer service representative can directly communicate with the user. During communication, the customer service representative can determine the cause of the abnormal event based on information of the abnormal event described by the user, and then enters the determined cause provided to the server. The server receives the cause entered by the customer service representative, determines the cause entered by the customer service representative as the cause of the abnormal event, sets an initial weight, and stores the obtained attribute information of the user, the cause entered by the customer service representative, and the initial weight in the first correspondence as a single record.", + "p0049": "For example, in this implementation of the present invention, the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight can be a decision tree model. For example, a root node in the decision tree model stores an abnormal event. The root node points to at least one level-2 node. Each level-2 node can store attribute information, and each level-2 node can point to at least one level-3 node. Each level-3 node can store a cause. A weight is set between each level-2 node and a level-3 node that the level-2 node points to.", + "p0050": "As shown in FIG. 2 , the root node stores an abnormal event X, the root node points to two level-2 nodes: a first level-2 node and a second level-2 node, the first level-2 node stores attribute information 1, and the second level-2 node stores attribute information 2.", + "p0051": "The first level-2 node points to three level-3 nodes: a first level-3 node, a second level-3 node, and a third level-3 node. The first level-3 node stores a cause A, the second level-3 node stores a cause B, and the third level-3 node stores a cause C.", + "p0052": "Weight 1 is set between the first level-2 node and the first level-3 node that the first level-2 node points to, weight 2 is set between the first level-2 node and the second level-3 node that the first level-2 node points to, and weight 3 is set between the first level-2 node and the third level-3 node that the first level-2 node points to.", + "p0053": "The second level-2 node points to three level-3 nodes: a fourth level-3 node, a fifth level-3 node, and a sixth level-3 node. The fourth level-3 node stores cause D, the fifth level-3 node stores cause E, and the sixth level-3 node stores cause F.", + "p0054": "Weight 4 is set between the second level-2 node and the fourth level-3 node that the second level-2 node points to, weight 5 is set between the second level-2 node and the fifth level-3 node that the second level-2 node points to, and weight 6 is set between the second level-2 node and the sixth level-3 node that the second level-2 node points to.", + "p0055": "Assume that when abnormal event X occurred, the attribute information of the user is the attribute information 1. In the decision tree model shown in FIG. 2 , starting from the root node, the level-2 nodes that the root node points to can be searched to identify a level-2 node that stores the attribute information 1, and it is identified that the level-2 node that stores the attribute information 1 is the first level-2 node.", + "p0056": "Because each level-3 node that the first level-2 node points to stores a different cause, starting from the first level-2 node, the level-3 nodes that the first level-2 node points to can be searched to identify a level-3 node that has the largest weight with the first level-2 node, and a cause stored in the identified level-3 node is determined as a cause of the abnormal event X. Assume that weight 2 is greater than weight 1, and weight 1 is greater than weight 3. The weight between the second level-3 node and the first level-2 node is the largest weight, cause B stored in the second level-3 node is then determined as the cause of the abnormal event X.", + "p0057": "In step S 105 , a cause in the record is determined as a cause of the abnormal event.", + "p0058": "If a plurality of records are selected in step S 104 , that is, in the first correspondence, there are a plurality of records that include the attribute information that have the largest weight, a record is randomly selected from the plurality of selected records, and a cause associated with the selected record is determined as the cause of the abnormal event. If a record is selected in step S 104 , a cause associated with the selected record is determined as the cause of the abnormal event.", + "p0059": "In the implementation of the present invention shown in FIG. 1 , the interaction data between the user and the server is obtained; whether an abnormal event occurred is detected based on the interaction data; when an abnormal event occurred, the attribute information of the user is obtained; and the locally stored first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight is searched to identify the record that includes the attribute information and the largest weight; and the cause in the record is determined as the cause of the abnormal event. In this process, the user does not need to communicate with the customer service representative, the customer service representative does not need to determine the cause of the abnormal event by communicating with the user, and the cause of the abnormal event can be determined by using the attribute information of the user and the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight. Therefore, labor costs can be saved.", + "p0060": "Generally, when the cause of the abnormal event is determined, help needs to be provided for the user to resolve the abnormal event. Therefore, in another implementation of the present invention, referring to FIG. 3 , the method further includes the following steps.", + "p0061": "In step S 201 , the abnormal event is processed based on the determined cause.", + "p0062": "In this implementation of the present invention, for the abnormal event, a technician collects common causes of the abnormal event in advance in a historical process, and a processing solution corresponding to each cause. For example, when the user fails to pay for an item online, the abnormal event is a payment failure. It is then determined that the cause of the abnormal event is that the balance in the user account is insufficient, and a preferred solution is to directly send a refill page used to refill the user account. As such, after receiving the refill page, the user can directly refill the user account with sufficient amount by using the refill page, and then payment can be successfully performed by using the amount, so as to resolve the abnormal event. For any causes of the abnormal event, the server can store the cause and a solution corresponding to the cause in a locally stored second correspondence between the cause of the abnormal event and a corresponding solution as a single record. For other causes of the event, previous operations can be similarly performed.", + "p0063": "Therefore, in this step, the second correspondence between the cause of the abnormal event and the corresponding solution can be obtained; a solution corresponding to the determined cause is obtained from the second correspondence; and the abnormal event is processed based on the obtained solution.", + "p0064": "In addition, in the previous implementation, a plurality of different abnormal events can sometimes be determined. In such case, the server needs to sequentially process these abnormal events. Technicians can set an abnormal event processing order in the server in advance, and the abnormal event processing order can store identifiers of the abnormal events in an order based on processing priorities.", + "p0065": "Therefore, in this implementation, when a plurality of different abnormal events are determined, the plurality of determined abnormal events can be determined in the predetermined abnormal event processing order; and the abnormal events are sequentially processed in an order obtained after sorting and based on causes of the abnormal events until the abnormal events are processed.", + "p0066": "After the server processes the abnormal event based on the determined cause, a processing result is obtained, which can be a processing success or a processing failure.", + "p0067": "For example, when the abnormal event is a payment failure, after the server processes the abnormal event, if the user can successfully perform payment during a retry, the processing result can indicate that the processing has succeeded. Or after the server processes the abnormal event, if the user still fails to perform payment during a retry, the processing result can indicate that processing has failed.", + "p0068": "In this implementation, the user can provide feedback to the processing result of the server. The feedback includes an agreement on the processing result of the abnormal event or a disagreement on the processing result.", + "p0069": "For example, after the server processes the abnormal event, if the user can successfully perform payment during a retry, the user can agree with the processing result obtained from the server that processed the abnormal event. After the server processes the abnormal event, if the user still fails to perform payment during a retry, the user can disagree with the processing result obtained from the server that processed the abnormal event.", + "p0070": "In still another implementation, to improve accuracy of subsequently determining the cause of the abnormal event by the server based on the first correspondence, the server can update the first correspondence based on the feedback from the user. Referring to FIG. 4 , the method further includes the following steps.", + "p0071": "In step S 301 , feedback data that is fed back by the user based on a processing result obtained from the server that has processed the abnormal event is received.", + "p0072": "The feedback data can include a predetermined flag bit. The predetermined flag bit can store an agreement identifier or a disagreement identifier. For example, \u201c1\u201d can indicate the agreement identifier, and \u201c0\u201d can indicate the disagreement identifier.", + "p0073": "When the user agrees with the processing result obtained from the server that has processed the abnormal event, the user can trigger, on the terminal device, an agreement button that indicates an agreement of the processing result. When the trigger operation of triggering the agreement button is received from the user, the terminal device generates the feedback data, adds the agreement identifier \u201c1\u201d to the predetermined flag bit in the feedback data, and then sends, to the server, the feedback data that the agreement identifier \u201c1\u201d is added to. After receiving the feedback data, the server determines whether an identifier included in the feedback data is the agreement identifier \u201c1\u201d or the disagreement identifier \u201c0\u201d, and when it is determined that the feedback data includes the agreement identifier \u201c1\u201d, the feedback data is determined to be agreement feedback data.", + "p0074": "In the previous implementation, when the server determines the cause of the abnormal event, the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight is searched to identify the record that includes the attribute information, the record that includes the largest weight is selected from the identified record that includes the attribute information. Then the cause associated with the record that includes the largest weight is used as the cause of the abnormal event. However, a real cause of the abnormal event may not be the cause associated with the record that includes the largest weight. When the cause is incorrectly determined, because a solution is obtained based on the determined cause, it cannot resolve the abnormal event.", + "p0075": "For example, the abnormal event is the payment failure, the attribute information does not include real-name authentication of the user account, and the user account is not bound to a bank card, and the cause associated with the record that includes the largest weight is that the balance in the account is insufficient, but the cause of the present abnormal event is that the payment function of the user account is disabled. A solution for cause from insufficient balance is to directly send the refill page used to refill the user account to the user. A solution for cause from disabled payment function is to provide to the user, an application page to apply for enabling the payment function. Therefore, the abnormal event of the present payment failure cannot be resolved by directly sending the refill page used to refill the user account to the user.", + "p0076": "Further, the user does not agree with the processing result obtained from the server that has processed the abnormal event based on the determined cause. When the user does not agree with the processing result obtained from the server that has processed the abnormal event based on the determined cause, the user can trigger, on the terminal device, a disagreement button used to indicate a disagreement on the processing result. When receiving the trigger operation of triggering the disagreement button by the user, the terminal device can generate the feedback data, add the disagreement identifier \u201c0\u201d to the predetermined flag bit in the feedback data, and send, to the server, the feedback data that the disagreement identifier \u201c0\u201d is added to. The server receives the feedback data, determines whether an identifier included in the feedback data is the agreement identifier \u201c1\u201d or the disagreement identifier \u201c0\u201d, and when it is determined that the feedback data includes the disagreement identifier \u201c0\u201d, determines that the feedback data is disagreement feedback data.", + "p0077": "In step S 302 , whether the feedback data includes an agreement identifier or a disagreement identifier is determined.", + "p0078": "When the feedback data includes the agreement identifier, in step S 303 , a weight in a record that includes the attribute information and the determined cause is added to the first correspondence.", + "p0079": "When the feedback data includes the agreement identifier, it is determined that the user agrees with the processing result obtained from the server that has processed the abnormal event based on the determined cause, the weight in the record that includes the attribute information, and the determined cause added to the first correspondence, so as to update the first correspondence based on the user feedback data.", + "p0080": "When the feedback data includes the disagreement identifier, in step S 304 , a cause of the abnormal event sent by a customer service representative is received. The received cause is sent after the customer service representative interacts with the user.", + "p0081": "When the feedback data includes the disagreement identifier, it is determined that the user does not agree with the processing result obtained from the server that has processed the abnormal event based on the determined cause, and it is likely that the cause of the abnormal event that is determined by the server in the previous implementation is not the real cause of the abnormal event. When the cause is incorrectly determined, because a solution is obtained based on the cause determined by the server, it cannot resolve the abnormal event.", + "p0082": "Therefore, to resolve the abnormal event, in this implementation of the present invention, the server can associate the connection between the server and the user's terminal device with the connection between the server and the customer service representative's terminal device, so that the user terminal device is directly connected to the customer service representative's terminal device. The customer service representative can then directly communicate with the user. During the communication, the customer service representative can determine the cause of the abnormal event based on information about the abnormal event described by the user, and then enters the determined cause provided to the server. The server receives the cause entered by the customer service representative. Generally, the cause of the abnormal event that is determined after the customer service representative communicates with the user is usually the real cause of the abnormal event, and therefore step S 305 can be performed.", + "p0083": "In still another implementation of the present invention, after receiving the cause entered by the customer service representative, the server can process the abnormal event based on the entered cause. Then, the user can continue to provide feedback based on a processing result obtained from the server that has processed the abnormal event based on the entered cause. When the server receives feedback data on this processing result from the user, if the feedback data includes the disagreement identifier, it indicates that the server still fails to help the user resolve the abnormal event, and the previous steps are repeatedly performed until feedback data that includes the agreement identifier is received from the user. If the feedback data includes the agreement identifier, it can be determined that the user agrees with the processing result obtained from the server that has processed the abnormal event based on the received cause, or it can be determined that the server has helped the user resolve the abnormal event, and step S 305 is performed.", + "p0084": "In step S 305 , whether the locally stored first correspondence includes a record that includes the attribute information and the received cause is determined.", + "p0085": "If the first correspondence includes the record that includes the attribute information and the received cause, a weight in the record that includes the attribute information and the received cause is added in step S 306 .", + "p0086": "If the first correspondence does not include the record that includes the attribute information and the received cause, in step S 307 , an initial weight is set, and the attribute information, the received cause, and the initial weight are stored in the first correspondence as a single record.", + "p0087": "FIG. 5 shows a data processing apparatus applicable to a server, based on an example implementation. Referring to FIG. 5 , the apparatus includes a first acquisition module 11 , configured to obtain interaction data between a user and the server; a detection module 12 , configured to detect, based on the interaction data, whether an abnormal event occurred; a second acquisition module 13 , configured to obtain attribute information of the user when an abnormal event occurred; a search module 14 , configured to search a locally stored first correspondence between attribute information corresponding to the abnormal event, a cause, and a weight for a record that includes the attribute information and a largest weight; and a determining module 15 , configured to determine a cause in the record as a cause of the abnormal event.", + "p0088": "In the implementation shown in FIG. 5 , the interaction data between the user and the server is obtained; whether an abnormal event occurred is detected based on the interaction data; when an abnormal event occurred, the attribute information of the user is obtained; and the locally stored first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight is searched to identify the record that includes the attribute information and the largest weight; and the cause in the record is determined as the cause of the abnormal event. In this process, the user does not need to communicate with a customer service representative, the customer service representative does not need to determine the cause of the abnormal event by communicating with the user, and the cause of the abnormal event can be determined by using the attribute information of the user and the first correspondence between the attribute information corresponding to the abnormal event, the cause, and the weight. Therefore, labor costs can be saved.", + "p0089": "Further, the apparatus can further include a processing module, configured to process the abnormal event based on the determined cause.", + "p0090": "If a plurality of abnormal events occurred, the processing module can include a sorting unit, configured to sort the plurality of abnormal events in a predetermined abnormal event order; and a first processing unit, configured to sequentially process, in an order of the sorted abnormal events, the abnormal events based on causes of the abnormal events until the abnormal events are processed.", + "p0091": "The processing module includes a first acquisition unit, configured to obtain a locally stored second correspondence between a cause corresponding to the abnormal event and a processing solution; a second acquisition unit, configured to obtain, from the second correspondence, a processing solution corresponding to the determined cause; and a second processing unit, configured to process the abnormal event based on the processing solution.", + "p0092": "Further, the apparatus further includes a first receiving module, configured to receive feedback data that is fed back by the user based on a processing result obtained from the server that has processed the abnormal event; and a first addition module, configured to add a weight in a record that includes the attribute information and the determined cause to the first correspondence, when the feedback data includes an agreement identifier; or a second receiving module, configured to receive a cause of the abnormal event that is sent by a customer service representative, when the feedback data includes a disagreement identifier, and where the received cause is sent after the customer service representative interacts with the user; and a second addition module, configured to add a weight in the record that includes the attribute information and the received cause, if the first correspondence includes a record that includes the attribute information and the received cause; or a storage module, configured to set an initial weight, and store the attribute information, the received cause, and the initial weight in the first correspondence as a single record, if the first correspondence does not include a record that includes the attribute information and the received cause.", + "p0093": "For the apparatus in the previous implementation, a specific manner in which each module performs an operation has been described in detail in the related method implementation, and repetitive details are omitted here for simplicity.", + "p0094": "A person skilled in the art can easily figure out another implementation solution of the present invention after considering the specification and practicing the present invention disclosed here. The present application is intended to cover any variations, functions, or adaptive changes. These variations, functions, or adaptive changes comply with general principles of the present invention, and can encompass common knowledge or a commonly used technical means in the technical field that are not disclosed in the present invention. The specification and the implementations are merely considered as examples, and the actual scope and the spirit of the present invention are pointed out by the appended claims.", + "p0095": "It should be understood that the present invention is not limited to the exact structures that are described above and that are shown in the accompanying drawings. Modifications and changes can be made without departing from the scope of the present invention. The scope of the present invention is defined only by the appended claims.", + "p0096": "FIG. 6 is a flowchart illustrating an example of a computer-implemented method for identifying potential solutions for abnormal events based on previous behavior, according to an implementation of the present disclosure. For clarity of presentation, the description that follows generally describes method 600 in the context of the other figures in this description. However, it will be understood that method 600 can be performed, for example, by any system, environment, software, and hardware, or a combination of systems, environments, software, and hardware, as appropriate. In some implementations, various steps of method 600 can be run in parallel, in combination, in loops, or in any order.", + "p0097": "At S 602 , the server obtains interaction data between client device and the server. In some implementations, the interaction data includes acquisition request data and acquisition response data. The client device transmits acquisition request data that indicates requested information from the user. For example, the acquisition request data can include a URL link to a website. The server receives the acquisition request data, retrieves a web page corresponding to the requested URL link, and provides an acquisition response that includes the web page to the client device. From S 602 , method 600 proceeds to S 604 .", + "p0098": "At S 604 , the server determines whether an abnormal event exists in the obtained interaction data. In some implementations, the server determines whether an abnormal event exists in the acquisition request data. For example, the acquisition request data that includes a requested URL link may have a spelling error. As a result, the server may not be able to respond with an appropriate or accurate web page in the acquisition response. This leads to an abnormal event in the obtained interaction data. In other implementations, the server determines whether an abnormal event exists in the acquisition response data. For example, the web page may not return properly, even with a correct URL link. Additionally, the server may not be able to communicate with the client device as an abnormal event. The server flags any of these examples as abnormal events from the obtained interaction data. From S 604 , method 600 proceeds to S 606 .", + "p0099": "At S 606 , in response to determining that the abnormal event exists in the obtained information data, the server obtains attribute information of a user corresponding to the client device from the obtained interaction data. The server obtains various attribute information of use that includes the user's gender, age, credentials of a user that include a username and password, whether the user account is tied to a bank card, whether real-name authentication is performed for the user account, whether a payment function of the user account is enabled, and client device specifics of the user, to name a few examples. The server obtains the attribute information and stores the attribute information in memory. From S 606 , method 600 proceeds to S 608 .", + "p0100": "At S 608 , the server identifies one or more records in a database that comprises the attribute information, a cause of the abnormal event, and a weight characterizing a severity of the abnormal event. In some implementations, the server scans through its database for any records that match and/or include the attribute information from the user. If the server finds any matches from its database that include the attribute information from the user, the server then retrieves a cause of an abnormal event and a weight characterizing the severity of that abnormal event from each of the retrieved records. From S 608 , method 600 proceeds to S 610 .", + "p0101": "At S 610 , the server determines a record from the one or more records identified in the database that includes a largest weight as the record that identifies a cause of the abnormal event. In some implementations, the server searches through a database of one or more records. In particular, the server compares the weights from each of the retrieved record and determines the record with the largest weight as the record to use for resolving the abnormal event. The weight value can indicate a number of times the abnormal event has occurred for the user of the corresponding attribute information. In other implementations, the weight can be assigned as a percentage as to the severity of the abnormal event. From S 610 , method 600 proceeds to S 612 .", + "p0102": "At S 612 , the server provides a processing solution to address the abnormal event based on the cause from the determined record. In particular, the server produces a solution to solve the issue caused by the abnormal event as described by the retrieved record with the largest weight. For example, the acquisition request may include a misspelled URL, and the acquisition response indicates that the web page is not successfully obtained. The server can produce a solution, after retrieving a record from the database that matches attribute information of the user that the URL in the acquisition request was incorrectly typed and to type in a correctly spelled URL. After S 612 , the method stops.", + "p0103": "The process described above is intended to obtain interaction data between a user and a server. If an abnormal event is detected in the interaction data, the attribute information of the user is obtained; and the locally stored first correspondence between the attribute information corresponding to the abnormal event, the cause of the abnormal event, and the weight characterizing the severity of the abnormal event is searched to identify the record that includes the obtained attribute information and the largest weight. The server retrieves the cause of the abnormal event in the record with the largest weight, and this cause is determined to be the cause of the abnormal event. In particular, the user does not need to communicate with the customer service representative regarding an issue of interactions between the user and the server. Additionally, the customer service representative does not need to determine the cause of the abnormal event by communicating with the user, and the cause of the abnormal event can be determined by using the attribute information of the user and the first correspondence between the attribute information corresponding to the abnormal event, the cause of the abnormal event, and the weight characterizing the severity of the abnormal event. As a result, this system automatically performs debugging of an abnormal event and providing a response to address the abnormal event. In addition, this system and methodology saves labor costs for the user.", + "p0104": "Embodiments and the operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification or in combinations of one or more of them. The operations can be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources. A data processing apparatus, computer, or computing device may encompass apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus can include special purpose logic circuitry, for example, a central processing unit , a field programmable gate array or an application-specific integrated circuit . The apparatus can also include code that creates an execution environment for the computer program in question, for example, code that constitutes processor firmware, a protocol stack, a database management system, an operating system , a cross-platform runtime environment, a virtual machine, or a combination of one or more of them. The apparatus and execution environment can realize various different computing model infrastructures, such as web services, distributed computing and grid computing infrastructures.", + "p0105": "A computer program can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A program can be stored in a portion of a file that holds other programs or data , in a single file dedicated to the program in question, or in multiple coordinated files . A computer program can be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.", + "p0106": "Processors for execution of a computer program include, by way of example, both general- and special-purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read-only memory or a random-access memory or both. The essential elements of a computer are a processor for performing actions in accordance with instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data. A computer can be embedded in another device, for example, a mobile device, a personal digital assistant , a game console, a Global Positioning System receiver, or a portable storage device. Devices suitable for storing computer program instructions and data include non-volatile memory, media and memory devices, including, by way of example, semiconductor memory devices, magnetic disks, and magneto-optical disks. The processor and the memory can be supplemented by, or incorporated in, special-purpose logic circuitry.", + "p0107": "Mobile devices can include handsets, user equipment , mobile telephones , tablets, wearable devices , implanted devices within the human body , or other types of mobile devices. The mobile devices can communicate wirelessly signals) to various communication networks . The mobile devices can include sensors for determining characteristics of the mobile device's current environment. The sensors can include cameras, microphones, proximity sensors, GPS sensors, motion sensors, accelerometers, ambient light sensors, moisture sensors, gyroscopes, compasses, barometers, fingerprint sensors, facial recognition systems, RF sensors , thermal sensors, or other types of sensors. For example, the cameras can include a forward- or rear-facing camera with movable or fixed lenses, a flash, an image sensor, and an image processor. The camera can be a megapixel camera capable of capturing details for facial and/or iris recognition. The camera along with a data processor and authentication information stored in memory or accessed remotely can form a facial recognition system. The facial recognition system or one-or-more sensors, for example, microphones, motion sensors, accelerometers, GPS sensors, or RF sensors, can be used for user authentication.", + "p0108": "To provide for interaction with a user, embodiments can be implemented on a computer having a display device and an input device, for example, a liquid crystal display or organic light-emitting diode /virtual-reality /augmented-reality display for displaying information to the user and a touchscreen, keyboard, and a pointing device by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, for example, visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.", + "p0109": "Embodiments can be implemented using computing devices interconnected by any form or medium of wireline or wireless digital data communication , for example, a communication network. Examples of interconnected devices are a client and a server generally remote from each other that typically interact through a communication network. A client, for example, a mobile device, can carry out transactions itself, with a server, or through a server, for example, performing buy, sell, pay, give, send, or loan transactions, or authorizing the same. Such transactions may be in real time such that an action and a response are temporally proximate; for example an individual perceives the action and the response occurring substantially simultaneously, the time difference for a response following the individual's action is less than 1 millisecond or less than 1 second , or the response is without intentional delay taking into account processing limitations of the system.", + "p0110": "Examples of communication networks include a local area network , a radio access network , a metropolitan area network , and a wide area network . The communication network can include all or a portion of the Internet, another communication network, or a combination of communication networks. Information can be transmitted on the communication network according to various protocols and standards, including Long Term Evolution , 5G, IEEE 802, Internet Protocol , or other protocols or combinations of protocols. The communication network can transmit voice, video, biometric, or authentication data, or other information between the connected computing devices.", + "p0111": "Features described as separate implementations may be implemented, in combination, in a single implementation, while features described as a single implementation may be implemented in multiple implementations, separately, or in any suitable sub-combination. Operations described and claimed in a particular order should not be understood as requiring that the particular order, nor that all illustrated operations must be performed . As appropriate, multitasking or parallel-processing can be performed.", + "features": { + "F1": "A computer-implemented method, performed by one or more processors, comprises obtaining interaction data between a client device and a server.", + "F2": "The method involves determining whether an abnormal event exists in the obtained interaction data by the one or more processors.", + "F3": "In response to determining that the abnormal event exists, the method includes obtaining attribute information of a user corresponding to the client device from the obtained interaction data.", + "F4": "The method identifies one or more records in a database, each record including the attribute information, a cause of one or more previous abnormal events, and a weight characterizing a severity of the previous abnormal events, using the one or more processors.", + "F5": "The method determines a cause of the abnormal event, where the determined cause is the same as the cause of the previous abnormal events from the record in the identified records that includes the largest weight.", + "F6": "The method provides a solution to address the abnormal event based on the determined cause.", + "F7": "The method determines an occurrence of a plurality of abnormal events in the obtained interaction data.", + "F8": "The method sorts the plurality of abnormal events in a predetermined abnormal event order.", + "F9": "The method executes the sorted plurality of abnormal events in a sequential order based on causes for each abnormal event.", + "F10": "The weight characterizing the severity of the abnormal event includes a quantity of times the abnormal event occurred.", + "F11": "The method includes executing the solution to address the abnormal event to produce a processing result.", + "F12": "The method involves receiving feedback data from a user of the client device based on the processing result obtained from the server after processing the abnormal event.", + "F13": "The feedback data comprises an indication of an agreement or disagreement on the executed solution as indicated by the user at the client device.", + "F14": "Identifying the records in the database further comprises comparing attribute information from each of the records in the database to the attribute information from the obtained interaction data.", + "F15": "In response to comparing the attribute information, the method determines a match between the attribute information from the records to the attribute information from the obtained interaction data.", + "F16": "A non-transitory, computer-readable medium stores instructions executable by a computer system to perform the operations of obtaining interaction data, determining abnormal events, and providing solutions.", + "F17": "A computer-implemented system comprises one or more computers and memory devices with instructions to perform the method of obtaining interaction data, determining abnormal events, identifying records, and providing solutions." + } + } + }, + { + "FAN": "106318129", + "CPC": [ + "A47C-031/00", + "A61B-005/00/22", + "A61B-005/0205", + "A61B-005/024", + "A61B-005/08", + "A61B-005/11", + "A61B-005/11/18", + "A61B-005/16", + "A61B-005/16/5", + "A61B-005/4806", + "A61B-005/4809", + "A61B-005/4812", + "A61B-005/486", + "A61B-005/7271", + "A61B-005/7405", + "A61B-005/742", + "A61B-2560/0242", + "A63F-013/212", + "G06F-016/00", + "G06F-016/2228", + "G06F-019/3418", + "G06F-019/3481", + "G06Q-020/10", + "G06Q-030/02", + "G06Q-030/06/41", + "G06Q-050/22", + "G16H-010/20", + "G16H-010/60", + "G16H-020/30", + "G16H-040/63", + "G16H-040/67", + "G16H-050/20", + "G16H-050/30", + "G16H-050/70", + "H04N-009/31/79", + "H04N-009/31/82", + "H04N-009/31/94" + ], + "Content": { + "title": "Information processing system, information processing server, storage medium storing information processing program, and information provision method", + "pa01": "An example information processing system performs an analysis relating to health of a user based on biological information of the user so as to provide information to the user based on a result of the analysis. The information processing system updates, for each user, at least a part of a rule for deciding information to be provided based on the biological information. Thus, the information processing system can provide information that is useful for the user. Note that the update of the rule is performed based on the biological information of the user that is obtained after the provision of the information, for example.", + "c-en-0001": "An information processing system, comprising: at least one processor; and a memory storing computer readable instructions that, when executed by the at least one processor, cause the information processing system to: use a rule that has been set for each user to decide content to be reproduced for aiding sleep of the user; reproduce the content decided based on the rule; perform an analysis relating to health of the user based on biological information of the user detected under an environment where the content is being reproduced; and update the rule so that the content is more likely to be reproduced when a result of the analysis indicates an improved health state of the user, the improved health state determined based on, at least, history of health information related to the sleep of the user, wherein the rule for selecting the content includes a first rule for deciding a specific content to be reproduced and a second rule for deciding a specific genre of the content to be reproduced, and the second rule for deciding the specific genre of the content to be reproduced is updated based on biological information of a plurality of users, and the first rule for deciding the specific content to be reproduced is updated for each user based on the biological information of the respective user and/or the second rule for deciding the specific genre of the content to be reproduced.", + "c-en-0002": "The information processing system according to claim 1 , the processor further configured to: obtain biological information of each of the plurality of users; and update the second rule based on at least a plurality of biological information included in the biological information of each of the plurality of users.", + "c-en-0003": "The information processing system according to claim 2 , the information processing system including one or more user terminals, and a server system that is configured to communicate with the one or more user terminals via a network; the one or more user terminals are configured to execute the analysis and the update of the rule; the one or more user terminals further configured to update the first rule based on the obtained biological information; the server system configured to obtain biological information of each of the plurality of users; and the information processing system configured to update the second rule based on the plurality of biological information obtained by the server.", + "c-en-0004": "The information processing system according to claim 1 , the processor further configured to update, for each user, at least a part of the rule based on the biological information of the user and an input by the user.", + "c-en-0005": "The information processing system according to claim 4 , the processor further configured to: present a question to the user; and use an answer to the question as the input by the user.", + "c-en-0006": "The information processing system according to claim 5 , the processor further configured to: decide content of the question based on the biological information of the user so as to present the question to the user; and update, for each user, at least a part of a rule based on the biological information of the user and the answer to the question.", + "c-en-0007": "The information processing system according to claim 1 , the processor further configured to: obtain environment information relating to an environment around the user when sensing the biological information of the user; and decide the content to be reproduced based on the biological information of the user and the environment information relating to the user.", + "c-en-0008": "The information processing system according to claim 1 , the processor further configured to: obtain environment information relating to an environment around the user when sensing the biological information of the user; and update at least a part of a rule based on the biological information of the user and the environment information relating to the user.", + "c-en-0009": "The information processing system according to claim 1 , the processor further configured to provide, to the user, information for improving the health of the user.", + "c-en-0010": "The information processing system according to claim 1 , the at least one processor of the information processing system configured to repeatedly execute the analysis based on biological information that is obtained repeatedly; the information processing system further comprising a storage medium configured to store at least a part of the biological information obtained repeatedly and information calculated in the repeatedly-executed analysis; and the at least one processor of the information processing system further configured to update, for each user, at least a part of the rule based on the biological information for a plurality of iterations and/or the information calculated over a plurality of iterations of the analysis, which are stored in the storage medium.", + "c-en-0011": "The information processing system according to claim 10 , the processor further configured to repeatedly update the rule.", + "c-en-0012": "The information processing system according to claim 11 , the processor further configured to update the rule each time the analysis is performed.", + "c-en-0013": "The information processing system according to claim 1 , wherein the biological information of the user is obtained from a sensor configured to sense at least one of pulse, breathing and body movements of the user.", + "c-en-0014": "The information processing system according to claim 1 , wherein the content is a composition or a movie for aiding sleep of the user.", + "c-en-0015": "The information processing system according to claim 14 , wherein the content is songs of a specific genre.", + "c-en-0016": "The information processing system according to claim 1 , wherein the result of the analysis indicates the improved health state when a time for the user to fall asleep after reproducing the content is short.", + "c-en-0017": "The information processing system according to claim 1 , wherein a weight value associated with the content is increased when the result of the analysis is positive.", + "c-en-0018": "The information processing system according to claim 1 , wherein the improved health state is determined based on health information for a plurality of sleep periods associated with the user.", + "c-en-0019": "The information processing system according to claim 1 , wherein the rule is updated based on detected health states of the plurality of users that are in a group associated with the user.", + "c-en-0020": "The information processing system according to claim 19 , wherein the group associated with the user, and including the plurality of users, is determined based on demographic information associated with the plurality of users.", + "c-en-0021": "The information processing system according to claim 1 , wherein a weight is assigned to the specific content based on the first rule, and the weight assigned to the specific content is corrected based on the second rule.", + "c-en-0022": "The information processing system according to claim 1 , wherein a weight is assigned to the specific content based on the first rule, and the weight is updatable using the second rule, and the system is configured to select the specific content using the first rule and without using the second rule in accordance with a user instruction.", + "c-en-0023": "The information processing system according to claim 1 , wherein the first rule is updated based on sleep time of the user, and the second rule is updated based on sleep latency of each user in the plurality of users.", + "c-en-0024": "A non-transitory storage medium storing an information processing program to be executed by a computer of an information processing device, the information processing program causing the computer to provide execution comprising: using a rule that has been set for each user to decide content to be reproduced for aiding sleep of the user; reproducing the content decided based on the rule; performing an analysis relating to health of the user based on biological information of the user detected under an environment where the content is being reproduced; and updating the rule so that the content is more likely to be reproduced when a result of the analysis indicates an improved health state of the user, the improved health state determined based on, at least, history of health information related to the sleep of the user, wherein the rule for selecting the content includes a first rule for deciding a specific content to be reproduced and a second rule for deciding a specific genre of the content to be reproduced, and the second rule for deciding the specific genre of the content to be reproduced is updated based on biological information of a plurality of users, and the first rule for deciding the specific content to be reproduced is updated for each user based on the biological information of the respective user and/or the second rule for deciding the specific genre of the content to be reproduced.", + "c-en-0025": "An information provision method to be executed by an information processing system, the information provision method comprising: using a rule that has been set for each user to decide content to be reproduced for aiding sleep of the user; reproducing the content decided based on the rule; performing an analysis relating to health of the user based on biological information of the user detected under an environment where the content is being reproduced; and updating the rule so that the content is more likely to be reproduced when a result of the analysis indicates an improved health state of the user, the improved health state determined based on, at least, history of health information related to the sleep of the user, wherein the rule for selecting the content includes a first rule for deciding a specific content to be reproduced and a second rule for deciding a specific genre of the content to be reproduced, and the second rule for deciding the specific genre of the content to be reproduced is updated based on biological information of a plurality of users, and the first rule for deciding the specific content to be reproduced is updated for each user based on the biological information of the respective user and/or the second rule for deciding the specific genre of the content to be reproduced.", + "p0001": "CROSS REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a continuation application of International Application No. PCT/JP2015/069653, filed on Jul. 8, 2015, which designated the U.S. and claims priority to International Application Nos. PCT/JP2014/070931 filed on Aug. 7, 2014, PCT/JP2014/078824 filed on Oct. 29, 2014, PCT/JP2014/078825 filed on Oct. 29, 2014, PCT/JP2014/078826 filed on Oct. 29, 2014, PCT/JP2014/078827 filed on Oct. 29, 2014, PCT/JP2014/078828 filed on Oct. 29, 2014, PCT/JP2014/078829 filed on Oct. 29, 2014, PCT/JP2015/061273 filed on Apr. 10, 2015 and PCT/JP2015/061274 filed on Apr. 10, 2015, the entire contents of each of which are hereby incorporated by reference.", + "p0003": "FIELD", + "p0004": "The present technique relates to an information processing system, an information processing server, a storage medium storing an information processing program and an information provision method for providing information from the server side to the terminal side.", + "p0005": "BACKGROUND AND SUMMARY", + "p0006": "There are conventional systems in which information obtained on the terminal side is uploaded to the server side, and the result of an analysis of the uploaded information is provided from the server side to the terminal side. For example, there is a system in which sleep data of a user is obtained by an evaluation device on the terminal side and the sleep data is analyzed on the server side so as to display the analysis result on a display device on the terminal side.", + "p0007": "In the prior art, it is possible that useful information is not provided, as the information to be provided from the server side to the terminal side.", + "p0008": "In view of this, the present application discloses an information processing system, an information processing server, a storage medium storing an information processing program and an information provision method capable of providing useful information.", + "p0010": "An example information processing system described herein includes analysis means and first update means. The analysis means is configured to perform an analysis relating to health of a user based on biological information of the user so as to provide information to the user based on a result of the analysis. The first update means is configured to update, for each user, based on the biological information of the user, at least a part of a rule for deciding information to be provided based on the biological information. The analysis means uses a rule that has been set for each user through updates to decide information to be provided to the user.", + "p0012": "The rule may include a first part that is updated for each user and a second part that is common among a plurality of users including the user.", + "p0014": "The information processing system may further include first obtaining means and second update means. The first obtaining means is configured to obtain biological information of each of a plurality of users. The second update means is configured to update the second part based on at least a plurality of biological information included in the biological information.", + "p0016": "The information processing system may include one or more user terminal, and a server system that is capable of communicating with the user terminal via a network. The user terminal may at least include the analysis means and the first update means. The first update means may update the first part based on the biological information obtained by the user terminal thereof. The server system may obtain biological information of each of the plurality of users. The second update means may update the second part based on the plurality of biological information obtained by the server.", + "p0018": "The first update means may update, for each user, at least a part of the rule based on the biological information of the user and an input by the user.", + "p0020": "The information processing system may further include question presenting means configured to present a question to the user. The first update means may use an answer to the question as the input by the user.", + "p0022": "The question presenting means may decide content of the question based on the biological information of the user that is obtained after the provision of the information so as to present the question to the user. The first update means may update, for each user, at least a part of the rule based on the biological information of the user that is obtained after the provision of the information and the answer to the question.", + "p0024": "The information processing system may further include second obtaining means configured to obtain environment information relating to an environment around the user when sensing the biological information. The analysis means may decide the information to be provided based on the biological information of the user and the environment information relating to the user.", + "p0026": "The information processing system may further include second obtaining means configured to obtain environment information relating to an environment around the user when sensing the biological information. The first update means may update at least a part of the rule based on the biological information of the user and the environment information relating to the user.", + "p0028": "The first update means may provide, to the user, information relating to health of the user.", + "p0030": "The first update means may provide, to the user, information for improving the health of the user.", + "p0032": "The analysis means may repeatedly execute the analysis based on biological information that is obtained repeatedly. The information processing system may further include memory control means configured to store, in a predetermined storage section, at least a part of the biological information obtained repeatedly and information calculated in the repeatedly-executed analysis. The first update means may update, for each user, at least a part of the rule based on the biological information for a plurality of iterations and/or the information calculated over a plurality of iterations of the analysis, which are stored in the storage section.", + "p0034": "The first update means may repeatedly update the rule.", + "p0036": "The first update means may update the rule each time the analysis means performs the analysis.", + "p0038": "The biological information may be obtained from a sensor configured to sense at least one of pulse, breathing and body movements of the user.", + "p0040": "The analysis means may perform an analysis relating to sleep and/or fatigue of the user based on the biological information.", + "p0042": "Another example information processing system described herein includes analysis means and information provision means. The analysis means is configured to perform an analysis relating to health of a user based on biological information of the user so as to decide information to be provided to the user based on a result of the analysis. The information provision means is configured to provide the decided information to the user. The analysis means decides the information to be provided to the user by using, as a rule for deciding the information to be provided, both a first rule that is common among a plurality of users and a second rule that is set individually for each of the plurality of users.", + "p0044": "The analysis means may calculate first information for deciding the information to be provided to the user by using the biological information and the first rule so as to decide the information to be provided to the user by using the calculated first information and the second rule.", + "p0046": "The analysis means may calculate second information by using the biological information and the second rule so as to decide the information to be provided to the user by correcting the first information with the second information.", + "p0047": "Note that the present specification discloses an example information processing device and an example server included in the information processing system as set forth in to above. The present specification also discloses a storage medium storing an information processing program that causes the computer of the information processing device or the server to function as some of the various units set forth in to above. The present specification also discloses an example information processing method to be carried out in the information processing system, the information processing device or the server.", + "p0048": "With the information processing system, the information processing device, the storage medium storing an information processing program and the information processing method set forth above, it is possible to provide information that is useful for the user.", + "p0049": "These and other objects, features, aspects and advantages will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.", + "p0050": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0051": "FIG. 1 is a block diagram illustrating an example of a configuration of an information processing system according to the first embodiment.", + "p0052": "FIG. 2 illustrates an example of a configuration of a terminal system.", + "p0053": "FIG. 3 illustrates an example of an arrangement of devices included in the terminal system.", + "p0054": "FIG. 4 is a functional block diagram illustrating an example of a functional configuration for calculating health information by a processing section of a main terminal device.", + "p0055": "FIG. 5 is a flow chart showing an example of a flow of a process executed by the processing section of the main terminal device.", + "p0056": "FIG. 6 is a timing chart showing an example of a flow of an operation performed by the information processing system.", + "p0057": "FIG. 7 illustrates an example of data stored in a data server.", + "p0058": "FIG. 8 illustrates an example of data stored in a portal server.", + "p0059": "FIG. 9 illustrates an example of a provision condition table.", + "p0060": "FIG. 10 illustrates an example of a use request.", + "p0061": "FIG. 11 illustrates an example of a use management table.", + "p0062": "FIG. 12 illustrates an example of user provision history information.", + "p0063": "FIG. 13 illustrates an example a main page displayed by the terminal system.", + "p0064": "FIG. 14 illustrates an example of an additional provision condition table.", + "p0065": "FIG. 15 illustrates an example of a method for determining an additional provision condition.", + "p0066": "FIG. 16 is a flow chart showing an example of a flow of a process executed by the data server.", + "p0067": "FIG. 17 is a flow chart showing an example of a flow of a process executed by a portal server.", + "p0068": "FIG. 18 illustrates an example of a condition update table.", + "p0069": "FIG. 19 illustrates an effect according to a variation.", + "p0070": "FIG. 20 illustrates an example of a provision condition table according to a variation.", + "p0071": "FIG. 21 illustrates an example of change information.", + "p0072": "FIG. 22 illustrates an example detailed configuration of the terminal system according to the second embodiment.", + "p0073": "FIG. 23 illustrates an example external appearance of the terminal system according to the second embodiment.", + "p0074": "FIG. 24 illustrates an example flow of a process performed by the terminal system according to the second embodiment.", + "p0075": "FIG. 25 illustrates an example image to be displayed on a hand-held terminal for providing information representing candidate tunes.", + "p0076": "FIG. 26 illustrates an example image to be displayed on the hand-held terminal at the time of awakening.", + "p0077": "FIG. 27 is a functional block diagram illustrating an example functional configuration of the terminal system.", + "p0078": "FIG. 28 illustrates an example genre decision rule.", + "p0079": "FIG. 29 illustrates an example table included in a tune decision rule.", + "p0080": "FIG. 30 illustrates an example table used for deciding a question to be presented to the user.", + "p0081": "FIG. 31 illustrates an example rule update table used for deciding the update content.", + "p0082": "FIG. 32 is a flow chart showing an example flow of a process executed on the hand-held terminal.", + "p0083": "FIG. 33 is a functional block diagram illustrating an example functional configuration of a terminal system according to a variation of the second embodiment.", + "p0084": "FIG. 34 is a functional block diagram illustrating an example functional configuration of a terminal system according to another variation of the second embodiment.", + "p0085": "DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS", + "p0086": "First Embodiment", + "p0088": "Hereinafter, an information processing system, an information processing server, an information processing program, and an information providing method according to the present embodiment will be described with reference to the drawings. Firstly, the overall configuration of the information processing system according to the present embodiment will be described. FIG. 1 is a block diagram illustrating an example of the configuration of the information processing system according to the present embodiment. As shown in FIG. 1 , the information processing system includes a terminal system 1 , a data server 2 , and a service server 3 . The system and servers 1 to 3 can communicate with each other through a network 4 such as the Internet and a mobile communication network.", + "p0089": "The terminal system 1 is disposed near a user . In FIG. 1 , the number of the terminal systems 1 is one. However, the information processing system includes a plurality of the terminal systems provided for users, respectively. In the present embodiment, the terminal system 1 obtains biological information of each user, and uploads the health information obtained from the biological information to the data server 2 . The biological information represents information sensed from a user's body. In the present embodiment, respiration, pulse, and body motion are obtained as the biological information. The health information represents information related to user's health and/or body. In the present embodiment, the health information includes a sleep index representing an index related to user's sleep, and a fatigue index representing an index related to fatigue of the user. Thus, the terminal system 1 senses the biological information of a user, calculates the sleep index and the fatigue index based on the result of sensing, and uploads the indexes to the data server 2 . The terminal system 1 senses a health state of a user, and can be also called a QOL sensor.", + "p0090": "The data server 2 stores the health information, of the user, which is transmitted from the terminal system 1 . Further, the data server 2 calculates statistics as secondary information obtained from the health information. In the following description, the health information received from the terminal system 1 and the secondary information obtained from the health information may be generically referred to as health information.", + "p0091": "The service server 3 uses the health information stored in the data server 2 to provide various information to the user of the terminal system 1 . In the following description, information provided to the user by the service server is referred to as \u201cprovision information\u201d. The provision information is information for improving an index indicated by the health information of a user. The provision information is, for example, information for introducing a product or a service for improving the index, and the details thereof will be described below. That is, in the present embodiment, the provision information may be also called recommendation information for a user. The service server 3 performs a service for providing the provision information by using the health information obtained from the data server 2 . Further, in the present embodiment, the service server 3 also provides a service for selling, to a user, a product and/or a service which is introduced by the provision information.", + "p0092": "Note that the data server 2 and the service server 3 each include one or more information processing devices having a CPU and a memory. On each of the servers 2 and 3 , the CPU executes an information processing program stored therein using the memory, thereby executing various information processes. In the present specification, the \u201cserver\u201d not only means one information processing device , but also means the entirety of a server device group when the server includes a plurality of server devices.", + "p0094": "Next, a configuration of the terminal system 1 according to the present embodiment will be described. FIG. 2 illustrates an example of the configuration of the terminal system 1 . FIG. 3 illustrates an exemplary case where devices included in the terminal system 1 are disposed. As shown in FIG. 2 and FIG. 3 , the terminal system 1 includes a main terminal device 10 and a sub-terminal device 20 . In the present embodiment, the main terminal device 10 and the sub-terminal device 20 are disposed in, for example, a use's bedroom . The terminal system 1 mainly obtains the biological information while the user is in bed , and calculates the sleep index and the fatigue index, and the details thereof will be described below.", + "p0095": "The main terminal device 10 functions as a sensor for sensing the biological information. In the present embodiment, as shown in FIG. 3 , the main terminal device 10 is disposed near a user such as at a user's bedside, and senses the biological information from the user who is in bed.", + "p0096": "As shown in FIG. 2 , the main terminal device 10 includes a Doppler sensor 11 which is an example of a sensor for sensing the biological information. The Doppler sensor 11 emits microwave and receives reflected wave of the emitted microwave, thereby sensing a moving body on the basis of a difference between a frequency of the emitted microwave and a frequency of the received microwave. An output waveform, from the Doppler sensor 11 , which represents the biological information having been sensed, is analyzed , whereby the biological information such as respiration, pulse, or body motion can be further calculated. The detail thereof will be described below. In the present embodiment, a non-worn type sensor capable of sensing the biological information in a state where the user does not wear the sensor is used, whereby the biological information can be sensed without hindering the user .", + "p0097": "The main terminal device 10 includes a camera 12 . The camera 12 is used to take an image of the user. Further, the main terminal device 10 includes a microphone 13 . The microphone 13 is used to sense, for example, sound from the user and/or ambient noise.", + "p0098": "The main terminal device 10 includes a processing section 15 for executing various information processing to be executed by the main terminal device 10 . The processing section 15 is connected to components 11 to 14 and 16 to 19 of the main terminal device 10 . The processing section 15 has a CPU and a memory, and the CPU uses the memory to execute an information processing program stored in the main terminal device 10 , whereby the various information processing described above is executed. In the present embodiment, the processing section 15 executes, as the information processing, a process of calculating the health information on the basis of the biological information sensed by the sensor, and the like. Further, when the main terminal device 10 has a function as the information processing device , the processing section 15 executes various information processing for the function.", + "p0099": "The main terminal device 10 includes an input/output interface, and functions as an information processing device for inputting and viewing of information. Specifically, the main terminal device 10 includes an operation input section 14 and a display 16 . The operation input section 14 is any input device that receives operation input from a user. In the present embodiment, the main terminal device 10 has, as the operation input section 14 , a button, and a touch panel provided on the display 16 . The display 16 is capable of displaying the biological information and/or the health information of a user, and displaying the provision information described above. Further, the display 16 may function as a mirror.", + "p0100": "The main terminal device 10 includes an illumination section 17 having a light source such as a LED. Light emission from the illumination section 17 is controlled by the processing section 15 . For example, the illumination section 17 may be controlled so as to appropriately emit light according to a sleep state while the user is sleeping, or emit light so as to act as an alarm clock for causing the user to wake up.", + "p0101": "The main terminal device 10 and the sub-terminal device 20 can communicate with each other. In an exemplary case, the main terminal device 10 has a short-range wireless communication section 18 as communication means for communicating with the sub-terminal device 20 . In the present embodiment, the short-range wireless communication section 18 is a communication module having a function of performing communication using a wireless LAN. For example, the short-range wireless communication section 18 is a communication module certified by Wi-Fi.", + "p0102": "Further, the main terminal device 10 is connected to the network 4 , and has a function of communicating with the servers 2 and 3 . Specifically, the main terminal device 10 includes a mobile communication section 19 . In the present embodiment, the mobile communication section 19 is a communication module that has a function of connecting to a mobile communication network and performing communication. For example, the mobile communication section 19 performs communication in a communication mode that complies with the 3G communication standard or the 4G ) communication standard. The method in which the main terminal device 10 communicates with the servers 2 and 3 may be a method in which communication is performed via a wireless LAN by a communication module certified by Wi-Fi. Further, the main terminal device 10 may have both a function of communicating with the servers 2 and 3 via a mobile communication network, and a function of communicating with the servers 2 and 3 via a wireless LAN.", + "p0103": "The sub-terminal device 20 functions as a sensor for obtaining biological information different from that obtained by the main terminal device 10 . As shown in FIG. 3 , the sub-terminal device 20 is disposed near the main terminal device 10 . The sub-terminal device 20 is capable of sensing the biological information of a user who has stepped thereon.", + "p0104": "As shown in FIG. 2 , the sub-terminal device 20 includes a load sensor 21 . The load sensor 21 senses a load applied to the top surface of the sub-terminal device 20 . The sub-terminal device 20 may have a sensor for sensing the biological information, other than a body weight, for measuring body fat or the like.", + "p0105": "Further, the sub-terminal device 20 includes a touch panel 22 . The touch panel 22 is disposed on the top surface of the sub-terminal device 20 . The touch panel 22 is capable of sensing the shape of a foot of a user who has stepped on the top surface of the sub-terminal device 20 .", + "p0106": "The sub-terminal device 20 includes a processing section 23 that executes various information processing to be executed by the sub-terminal device 20 . The processing section 23 has a CPU and a memory, and the CPU uses the memory to execute an information processing program stored in the sub-terminal device 20 , thereby executing various information processing.", + "p0107": "The sub-terminal device 20 includes a short-range wireless communication section 24 as communication means for communicating with the main terminal device 10 . Similarly to the short-range wireless communication section 18 of the main terminal device 10 , the short-range wireless communication section 24 is a communication module that has a function of performing communication using a wireless LAN, and is, for example, a communication module certified by Wi-Fi. In the present embodiment, when it is sensed that a user has stepped on the sub-terminal device 20 , the short-range wireless communication section 24 transmits a predetermined notification signal and the biological information sensed by the load sensor 21 , from the sub-terminal device 20 , to the main terminal device 10 .", + "p0109": "Next, a process operation executed by the terminal system 1 will be described. In the present embodiment, the main terminal device 10 operates in three kinds of modes, that is, in a normal mode, a sensing mode, and a sleep mode. In the normal mode, the main terminal device 10 is used as an input/output terminal, and receives an operation input from a user, executes processing according to the operation input, and displays an image of a result of the processing on the display 16 . In the sensing mode, the biological information of a user who is in bed is sensed, and the health information is calculated. In the sleep mode, the above-described processes to be executed in the normal mode and the sensing mode are not executed, and power consumption is reduced. However, also in the sleep mode, the main terminal device 10 is allowed to receive a signal from the sub-terminal device 20 .", + "p0110": "The sub-terminal device 20 is capable of constantly performing sensing as to whether or not a user has stepped on the sub-terminal device 20 . That is, in the sub-terminal device 20 , at least one of the load sensor 21 and the touch panel 22 is constantly actuated, to sense that a user has stepped on the sub-terminal device 20 . In other embodiments, the sub-terminal device 20 may be operable in a sleep mode and an ON mode . In the sleep mode, the processing section 23 does not operate, and at least one of the load sensor 21 and the touch panel 22 operates. In the ON mode, at least the processing section 23 operates. In this case, when the load sensor 21 and/or the touch panel 22 sense that a user has stepped on the sub-terminal device 20 in a state where the sub-terminal device 20 is in the sleep mode, the mode may be shifted to the ON mode.", + "p0112": "Hereinafter, processing performed when the terminal system 1 operates in the sensing mode while a user is in bed, will be described. It is assumed that, before the user goes to bed, the main terminal device 10 is in the sleep mode or the normal mode. At this time, when the user has stepped on the sub-terminal device 20 shown in FIG. 3 ), the processing section 23 of the sub-terminal device 20 senses that the user has stepped on the sub-terminal device 20 , and measures the body weight on the basis of a result of sensing from the load sensor 21 . Further, the processing section 23 calculates the shape of a foot of the user on the basis of a result of sensing from the touch panel 22 . In other embodiments, the sub-terminal device 20 may include a camera, and the processing section 23 may calculate the shape of the foot on the basis of an image taken by the camera.", + "p0113": "In the present embodiment, the processing section 23 performs authentication of the user by using the calculated shape of the foot shown in FIG. 3 ). That is, data indicating the shape of the foot of an authenticated user is previously registered in the sub-terminal device 20 , and the processing section 23 determines whether or not the calculated shape of the foot and the registered shape of the foot coincide with each other, thereby performing authentication. When it is determined that the two shapes of the feet coincide with each other, the processing section 23 determines that the authentication has succeeded. In this case, the processing section 23 transmits a predetermined authentication notification to the main terminal device 10 shown in FIG. 3 ). The processing section 23 may transmit, to the main terminal device 10 , both the authentication notification and the biological information sensed by the sub-terminal device 20 . Meanwhile, when it is determined that the two shapes of the feet do not coincide with each other, the processing section 23 determines that authentication has failed. In this case, the processing section 23 does not transmit the authentication notification to the main terminal device 10 .", + "p0114": "When the main terminal device 10 receives the authentication notification, the main terminal device 10 shifts from the sleep mode or the normal mode to the sensing mode shown in FIG. 3 ). Thereafter, the biological information of the user is obtained by the main terminal device 10 .", + "p0115": "When the sensing mode is started, the processing section 15 of the main terminal device 10 firstly actuates a sensor for sensing the biological information, and starts obtaining a result of sensing from the sensor. When the camera 12 and/or the microphone 13 are used as the sensor for sensing, the processing section 15 may actuate the camera 12 and/or the microphone 13 . In the sensing mode, the Doppler sensor 11 continuously performs the sensing, and continuously outputs an output waveform as the result of sensing. The processing section 15 calculates the health information based on the result of sensing from the Doppler sensor 11 .", + "p0117": "Hereinafter, the process of calculating the health information on the basis of the result of sensing from the Doppler sensor 11 will be described. FIG. 4 is a functional block diagram illustrating an example of a functional configuration for calculating the health information by the processing section 15 . As shown in FIG. 4 , the processing section 15 includes a waveform analysis section 31 , a sleep calculation section 32 , an autonomic nerve calculation section 33 , and a fatigue calculation section 34 .", + "p0118": "The waveform analysis section 31 calculates respiration, pulse, and body motion as further biological information on the basis of the biological information sensed by the Doppler sensor 11 . To date, it has been known that the output waveform from the Doppler sensor 11 is separated according to frequencies, whereby waveforms representing respiration, pulse, and body motion can be obtained. The waveform analysis section 31 separates the output waveform into a frequency band corresponding to the respiration, a frequency band corresponding to the pulse, and a frequency band corresponding to the body motion, by performing frequency analysis and the like, and outputs the data of each waveform obtained by the separation. As shown in FIG. 4 , the output from the waveform analysis section 31 is inputted to each of the sleep calculation section 32 and autonomic nerve calculation section 33 .", + "p0119": "The sleep calculation section 32 calculates various sleep indexes on the basis of the biological information . To date, a method for calculating sleep indexes on the basis of respiration, pulse, and body motion has been known. In the present embodiment, the sleep calculation section 32 calculates sleep indexes indicating the following information.", + "p0120": "sleep latency", + "p0121": "WASO", + "p0122": "number of mid-sleep awakenings", + "p0123": "sleep efficiency", + "p0124": "total sleep time", + "p0125": "activity level during sleep", + "p0126": "sleep stage", + "p0127": "REM sleep time", + "p0128": "non-REM sleep time", + "p0129": "sleep quality", + "p0130": "In other embodiments, a portion of the sleep indexes described above may be calculated, or different kinds of sleep indexes other than the sleep indexes descried above may be calculated.", + "p0131": "The autonomic nerve calculation section 33 calculates an index indicating an action level of autonomic nerves on the basis of the biological information. Specifically, a waveform of a pulse included in the biological information is subjected to frequency analysis by using the maximum entropy method and Fourier transformation, and a high-frequency component HF and a low-frequency component LF of the waveform are calculated. It has been known that the high-frequency component HF represents an action level of parasympathetic nerves, and the low-frequency component LF represents an action level of sympathetic nerves. Further, it has been known that a fatigue level can be evaluated on the basis of a ratio between an action level of parasympathetic nerves and an action level of sympathetic nerves . Therefore, the autonomic nerve calculation section 33 calculates, as the autonomic nerve index, a ratio of the low-frequency component LF to the high-frequency component HF. As shown in FIG. 4 , the output from the autonomic nerve calculation section 33 is used as an input to the fatigue calculation section 34 .", + "p0132": "The fatigue calculation section 34 calculates a fatigue index on the basis of the sleep index and the autonomic nerve index. In the present embodiment, as the fatigue index, the degrees of fatigue represented as five stages from fatigue degree 1 to fatigue degree 5 are calculated. There is no particular limitation on the method for calculating the fatigue index. For example, the following method can be used.", + "p0133": "A first method is a method in which a fatigue index is calculated according to a sleep index. The sleep index is considered to have a correlation with a fatigue level. For example, an exemplary case in which the fatigue level is assumed to be high is as follows.", + "p0134": "The sleep latency is long.", + "p0135": "WASO is long.", + "p0136": "The number of mid-sleep awakenings is high.", + "p0137": "The sleep efficiency is poor.", + "p0138": "The total sleep time is short.", + "p0139": "The balance between the REM sleep hours and the non-REM sleep hours is poor .", + "p0140": "Therefore, the fatigue calculation section 34 calculates the fatigue level so as to increase the fatigue level when the sleep index represents the above-described case, and reduce the fatigue level when the sleep index does not represent the above-described case. For example, the fatigue calculation section 34 may determine whether or not the above-described items are satisfied, calculate a point based on the number of satisfied items, and calculate the fatigue level on the basis of the total point. At this time, the fatigue calculation section 34 may perform weighting for each item to calculate the point. Alternatively, a reference value may be set for each item, and the point may be calculated such that the greater a difference between the calculated value of the sleep index and the reference value is, the greater the point is.", + "p0141": "As described above, in the present embodiment, the sleep index for user's sleep is calculated on the basis of the biological information, and the fatigue index is calculated on the basis of the sleep index. As described above, since it is considered that there is a correlation between the sleep index and the fatigue degree, the fatigue index is calculated on the basis of the sleep index, whereby the fatigue index can be obtained with an enhanced accuracy.", + "p0142": "A second method is a method in which a fatigue level is calculated on the basis of sleep hours in a predetermined time period . To date, there has been a method for calculating a fatigue level on the basis of sleep hours and a working time according to Fatigue Risk Management System . In this method, for example, if the working time is assumed to be constant for simplicity, the fatigue level can be calculated on the basis of the sleep hours.", + "p0143": "A third method is a method for calculating a fatigue level on the basis of the autonomic nerve index. As described above, it has been known that a fatigue level can be evaluated by using a balance between an action level of sympathetic nerves and an action level of parasympathetic nerves, that is, by using the autonomic nerve index described above. Therefore, the fatigue calculation section 34 calculates a fatigue level such that, for example, the greater a difference between the reference value and a value of the autonomic nerve index is, the higher the fatigue level is.", + "p0144": "In the present embodiment, the fatigue calculation section 34 calculates a fatigue level by using the above three methods. Specifically, the fatigue calculation section 34 calculates the degrees of fatigue in the above three methods, and calculates a final fatigue level on the basis of the calculated degrees of fatigue. For example, the fatigue calculation section 34 may use an average of the three values of the fatigue level as the final fatigue level, or may calculate the final fatigue level by performing weighting of any one of the three values of the fatigue level.", + "p0145": "In other embodiments, there is no particular limitation on the method for calculating the fatigue index, and the content of the fatigue index may be also optionally set. In other embodiments, a value indicating a fatigue degree for each fatigue type may be calculated as the fatigue index. For example, in another embodiment, the fatigue index may represent three kinds of values that are a value representing an acute-fatigue degree, a value representing a cumulative-fatigue degree, and a value representing mental-fatigue degree.", + "p0146": "In other embodiments, the health information may be calculated by using a result of sensing from the camera 12 and/or the microphone 13 . For example, the biological information such as pulse and/or body motion may be calculated on the basis of an image of a user which is taken by the camera 12 . Therefore, the processing section 15 may calculate the sleep index by using the biological information obtained from an image taken by the camera 12 in addition to the biological information obtained from a result of sensing from the Doppler sensor 11 . Further, the processing section 15 may calculate the sleep index in consideration of snore sensed by the microphone 13 .", + "p0147": "FIG. 5 is a flow chart showing an example of a flow of a process executed by the processing section 15 of the main terminal device 10 . FIG. 5 mainly shows the process in the sensing mode, and the processes in the normal mode and the sleep mode are not shown.", + "p0148": "In the present application, the processes of steps in the flow charts shown in the drawings are merely examples, and the order in which the steps are performed in the process may be changed or another process in addition to the process of the steps may be executed when the same result is obtained. In the description herein, the CPU of the processing section executes the process of the steps in the flow chart. A processor or a dedicated circuit other than the CPU may execute the process of a part of the steps in the flow chart.", + "p0149": "In step S 1 shown in FIG. 5 , the processing section 15 confirms reception of data from the sub-terminal device 20 . At a time when step S 1 is executed, the main terminal device 10 is in the sleep mode or the normal mode. In the sleep mode or the normal mode, the processing section 15 repeatedly executes the process steps of step S 1 once in every predetermined time period.", + "p0150": "In step S 2 , the processing section 15 determines whether or not the mode is to be shifted to the sensing mode. That is, the processing section 15 determines that mode is to be shifted to the sensing mode when the above-described authentication notification is received from the sub-terminal device 20 in step S 1 , and determines that the mode is not shifted to the sensing mode when the authentication notification is not received. When the result of the determination in step S 2 is affirmative, the process step of step S 3 is executed. Meanwhile, when the result of determination in step S 2 is negative, the process step of step S 1 is executed again.", + "p0151": "In step S 3 , the processing section 15 shifts the operation mode of the main terminal device 10 to the sensing mode. That is, the processing section 15 actuates the Doppler sensor 11 . In the following step S 4 , the processing section 15 obtains a result of sensing from the Doppler sensor 11 . Further, in the following step S 5 , the processing section 15 calculates, as the further biological information, respiration, pulse, and body motion on the basis of the result of sensing. In the sensing mode, a series of process steps from step S 4 to S 6 is repeatedly executed once in every predetermined time period. Therefore, in the sensing mode, the biological information is repeatedly obtained and is successively stored in a memory of the processing section 15 .", + "p0152": "In step S 6 , the processing section 15 determines whether or not the sensing mode is to be ended. In the present embodiment, whether or not the sensing mode is to be ended is determined according to whether or not a user is in bed. Specifically, the processing section 15 determines whether or not the user is outside the range for sensing by the Doppler sensor 11 , on the basis of the result of sensing from the Doppler sensor 11 . When the user is outside the range, the processing section 15 determines that the user is not in bed, and the sensing mode is to be ended. Meanwhile, when the user is in the range for the sensing by the Doppler sensor 11 , the processing section 15 determines that the user is in bed, and the sensing mode is not to be ended. When a result of the determination in step S 6 is affirmative, the process step of step S 7 is executed. Meanwhile, when a result of the determination in step S 6 is negative, the process step of step S 4 is executed again.", + "p0153": "In step S 7 , the processing section 15 calculates the health information on the basis of the result of sensing from the Doppler sensor 11 . That is, as described above in \u201c \u201d, the sleep index and the fatigue index are calculated on the basis of the biological information. In the following step S 8 , the processing section 15 transmits, to the data server 2 , the health information calculated in step S 7 . Thus, in the present embodiment, the health information is automatically generated and transmitted to the data server 2 . In other embodiments, information to be transmitted from the terminal system 1 to the data server 2 may be transmitted according to a user performing an operation for transmission. Furthermore, the processing section 15 may transmit, to the data server 2 , both the health information, and the information received together with the authentication notification from the sub-terminal device 20 .", + "p0154": "In step S 9 , the processing section 15 shifts the operation mode of the main terminal device 10 to the sleep mode. That is, the processing section 15 halts the operation of the Doppler sensor 11 . After the operation mode has been shifted to the sleep mode in step S 9 , the process step of step S 1 described above is executed again.", + "p0155": "As described above, in the present embodiment, the process of calculating the health information on the basis of the biological information that is a result of sensing from the sensor is executed on the terminal side. However, this process may be shared by the terminal side and the server side in any manner. For example, in another embodiment, the terminal system 1 may transmit the biological information to the data server 2 , and the data server 2 may calculate the health information on the basis of the biological information. At this time, some health information may be calculated also by the terminal system 1 , and may be displayed on the display 16 at an appropriate time. Thus, the terminal system 1 can provide the health information to the user without obtaining data from the data server 2 . Further, the health information calculated by the data server 2 may be transmitted to the terminal system 1 and displayed by the terminal system 1 .", + "p0156": "In the present embodiment, the health information is transmitted from the terminal system 1 to the data server 2 when the sensing mode ends. In other embodiments, the transmission information to be transmitted from the terminal system 1 to the data server 2 may be successively transmitted during the sensing mode.", + "p0158": "In the present embodiment, the main terminal device 10 shifts to the normal mode according to a user performing a predetermined operation in the sleep mode. The main terminal device 10 accesses the service server 3 and displays the health information of the user on the display 16 or displays, on the display 16 , the provision information provided by the service server 3 in the normal mode, and the detail thereof will be described below. Further, the main terminal device 10 shifts to the sleep mode according to a user performing a predetermined operation in the normal mode.", + "p0160": "As described above, in the present embodiment, the terminal system includes: a first device having a first sensor for sensing first biological information for a user: and a second device having a second sensor for sensing second biological information that is the biological information for the user and is different from the first biological information. Further, the terminal system is capable of communicating with a server for performing analysis based on the first biological information and the second biological information.", + "p0161": "The terminal system transmits, to the server , transmission information that includes at least one of: the first biological information and the second biological information; and the health information obtained from the biological information. The first biological information and the second biological information may be transmitted individually from different devices, respectively. The server performs analysis based on the transmission information transmitted from the terminal system.", + "p0162": "Thus, the terminal system is capable of obtaining a lot of the biological information from two devices, thereby improving the quality of analysis by the server. That is, the server is allowed to perform increased kinds of analyses or perform analysis with enhanced accuracy.", + "p0163": "In the present embodiment, the first sensor senses the biological information of a user who is in bed, and the second sensor senses the biological information of a user who stays awake . Thus, the biological information can be obtained both while the user is in bed and during the other period, thereby improving the quality of analysis by the server.", + "p0164": "In the present embodiment, the first sensor is a non-contact type sensor capable of sensing the biological information in a state where a user is not in contact with the sensor, and the second sensor is a contact-type sensor capable of sensing the biological information in a state where a user is in contact with the sensor. Thus, the biological information can be sensed without making the user conscious of the sensing while the user is in bed, and the biological information can be assuredly sensed while the user stays awake.", + "p0165": "In the present embodiment, the first device has the following means.", + "p0166": "Reception means for receiving, from the second device, the second biological information sensed by the second sensor", + "p0167": "Transmission means for transmitting, to the server , transmission information that includes at least one of: the first biological information sensed by the first sensor and the second biological information received from the second device; and the health information based on the above biological information.", + "p0168": "In the above configuration, the biological information sensed by the two devices can be efficiently transmitted to the server. Further, the second device may not have a function of communicating with the server, thereby simplifying the configuration of the device.", + "p0169": "In the present embodiment, the server transmits, to the terminal system, the information based on a result of analysis, and the detail thereof will be described below. The first device includes: reception means for receiving information transmitted from the server; and display means for displaying the received information. Thus, the user is allowed to easily view the above information by using the first device. Further, the above information is displayed by the first device that senses the biological information, thereby simplifying the configuration of the terminal system.", + "p0170": "In other embodiments, the terminal system 1 may have a device for receiving and displaying the provision information, separately from the main terminal device 10 . For example, the information terminal may be any information processing device, such as a mobile terminal, a personal computer, and a game apparatus, used by a user. For example, the information terminal of the user may access the service server 3 , and receive and display the provision information. Further, purchasing of the product/service described below may be performed by using the information terminal.", + "p0171": "In the present embodiment, the second device transmits a predetermined notification to the first device according to the second biological information being sensed. The first device starts sensing the first biological information according to the predetermined notification being received. Thus, the first device can efficiently execute the operation of sensing the biological information, and information processing can be efficiently performed by the first device. Further, power consumption in the first device can be reduced.", + "p0172": "In the present embodiment, the second device executes, according to the second biological information being sensed, the authentication process for the user by using the biological information of the user. When the authentication has succeeded , the second device transmits the predetermined notification to the first device. Thus, when the user is authenticated as one of the registered users, the first device starts sensing the biological information, whereby the sensing process can be efficiently executed. When the number of registered users is plural, a user for whom sensing is to be performed can be identified. The biological information used in the authentication process may be the second biological information or other biological information.", + "p0173": "In the present embodiment, the main terminal device 10 starts measuring the biological information if the authentication has succeeded in the sub-terminal device 20 , and ends measurement of the biological information if the biological information is not sensed . In other embodiments, there is no particular limitation on the method for determining a period in which the biological information is measured, and the measurement period may be determined in another method.", + "p0174": "For example, in another embodiment, in the main terminal device 10 , sensing may be intermittently performed by the sensor and the measurement period may be determined on the basis of the result of sensing. Specifically, the main terminal device 10 determines whether or not a user is sensed , by the Doppler sensor 11 performing sensing at predetermined time intervals. When a user is not sensed, the main terminal device 10 halts sensing by the Doppler sensor 11 . In this case, the measurement is not started. Meanwhile, when a user is sensed, the main terminal device 10 continues the sensing by the Doppler sensor 11 , thereby starting the measurement. Further, when the measurement has been started, the main terminal device 10 continues the measurement while a user is sensed by the Doppler sensor 11 . That is, similarly to the above embodiment, the main terminal device 10 ends the measurement by the Doppler sensor 11 according to the user being not sensed. Thus, the period for the measurement by the Doppler sensor 11 can be determined on the basis of a result of sensing by the Doppler sensor 11 itself. Thus, the measurement period can be determined without using a device such as another sensor, thereby simplifying the configuration of the device.", + "p0175": "In other embodiments, the period for the measurement by the Doppler sensor 11 may be determined on the basis of a result of sensing from a sensor different from the Doppler sensor 11 . For example, the terminal system 1 may use an infrared sensor and/or a camera as the human sensor. Specifically, the terminal system 1 intermittently or continuously senses a user by the human sensor. In a period in which a user is sensed by the human sensor, the terminal system 1 performs the measurement by the Doppler sensor 11 . Also in this manner, the period for the measurement can be automatically determined, similarly to the method of the present embodiment, and to a method for determining a period for the measurement on the basis of a result of sensing by the Doppler sensor 11 . That is, since a user need not perform an operation for measurement, usability of the terminal system 1 can be improved. Furthermore, the biological information can be measured without causing a user to spend time and putting a burden on the user. Therefore, continuous obtaining of the biological information is facilitated.", + "p0176": "In other embodiments, the period for the measurement by the Doppler sensor 11 may be previously determined. For example, the terminal system 1 may perform the measurement in a predetermined time slot , or the measurement may be constantly performed . Also in this manner, similarly to the above manner, the user need not perform an operation for starting and ending the measurement, whereby usability of the terminal system 1 can be improved.", + "p0177": "In the above embodiment, the terminal system 1 automatically calculates the health information on the basis of the biological information in addition to the biological information being automatically measured. Therefore, even if a user does not perform a specific operation, the health information is calculated by the terminal system 1 , whereby usability of the terminal system 1 can be improved.", + "p0178": "In the present embodiment, the first device that is a single device includes first communication means for communicating with the server via a mobile telephone communication network, and the second communication means for performing short-range wireless communication with the second device. The first device is capable of easily communicating with the server and the second device.", + "p0179": "As described above, in the present embodiment, the terminal system 1 uses a non-contact type sensor , whereby a user need not wear a sensor. Since the biological information is automatically measured while the user is in bed, the user need not wait for measurement of the biological information, and need not perform a bothersome operation for measurement. Further, when the sensor is disposed near the user who is in bed, a bothersome setting operation need not be performed for each measurement. Thus, the terminal system 1 can measure the biological information without causing the user to spend time and putting a burden on the user, whereby continuous obtaining of the biological information is facilitated.", + "p0180": "The terminal system 1 may reproduce content for inducing sleep onset or wake-up of a user. For example, the main terminal device 10 may reproduce music for inducing sleep onset of the user according to start of sensing by the Doppler sensor 11 . At this time, the main terminal device 10 may control reproduction of the content according to a sleep state of the user. Thus, the content can be reproduced in an appropriate method based on the sleep state of the user, and the content can be effectively reproduced. Reproduction of the content is halted in a state where the effect seem to be small , thereby reducing power consumption. Hereinafter, a specific exemplary case will be described.", + "p0181": "The main terminal device 10 determines a sleep state of a user on the basis of a result of sensing from the Doppler sensor 11 while the user is sleeping. For example, the main terminal device 10 determines, in real time, whether or not the user is in a sleep state, and determines, in real time, the depth of sleep . The \u201cdetermines in real time\u201d described above strictly means that determination is immediately performed and also means that the determination is performed with about several seconds delay.", + "p0182": "The main terminal device 10 controls reproduction of the content according to the sleep state, of the user, which is determined while the user is sleeping. The main terminal device 10 may control reproduction of the content/halting of reproduction of the content according to the sleep state, or may control the reproduction method according to the sleep state. For example, the main terminal device 10 may start reproduction of the content for causing a user to sleep before the user sleeps, and halt the reproduction of the content according to the user having fell asleep. Alternatively, the main terminal device 10 gradually reduces a sound volume for reproduction of the content according to the user having fell asleep, and halts reproduction of the content according to the user having been deep in sleep . Further, for example, the main terminal device 10 may start reproduction of content according to a time when the user has awakened, or may start reproduction of content according to a time when the user is assumed to awaken soon.", + "p0183": "In the terminal system 1 , the content reproduced when the user sleeps may be stored in the terminal system 1 or may be obtained from an external device . For example, the service server 3 may transmit the content to the terminal system 1 in a providing process for providing the product/service .", + "p0185": "Next, the process operation executed by the information processing system will be described. FIG. 6 is a timing chart showing an example of a flow of an operation performed by the information processing system. Hereinafter, a process operation performed when the information processing system generally performs the following operation will be described as an exemplary process operation.", + "p0186": "<1> The health information obtained by the terminal system 1 is repeatedly transmitted to the data server 2 , and stored and accumulated.", + "p0187": "<2> The service server 3 generates provision information for introducing a product/service, by using the health information that is accumulated in the data server 2 , and provides the provision information to the terminal system 1 .", + "p0188": "<3> A user who views the provision information in the terminal system 1 , purchases the product/service introduced by the provision information of the product/service when the product/service is available at no charge).", + "p0189": "<4> After the product/service is purchased, the service server 3 generates additional provision information by using the health information accumulated in the data server 2 , and provides the additional provision information to the terminal system 1 .", + "p0190": "In the present embodiment, the service server 3 includes a portal server 41 and a service providing server 42 . The portal server 41 is a server that manages a portal site for providing, to the user, the health information accumulated in the data server 2 , or the provision information described above. The portal server 41 has a function of accessing the data server 2 , generating the provision information by using the health information accumulated in the data server 2 , and providing the provision information to the user .", + "p0191": "The service providing server 42 is a server that executes a process for providing the product/service to a user. In the present embodiment, purchasing of the product/service provided by the service providing server 42 is managed by the portal server 41 . That is, the portal site functions as shopping site for selling the product/service provided by the service providing server 42 . A user accesses the portal site by using the terminal system 1 and purchases the product/service on the portal site. The service providing server 42 is typically managed by a service provider that provides individual product/service, and the portal server 41 is managed by a management company that manages the data server 2 . The service providing server 42 may include a plurality of servers for, for example, respective service providers.", + "p0192": "<1> Process of Accumulating Health Information in Data Server", + "p0193": "As described above in \u201c \u201d, the terminal system 1 transmits the health information to the data server 2 . In the present embodiment, the terminal system 1 transmits both the health information and user identification information that is preset for each user. The terminal system 1 continuously transmits the health information periodically to the data server 2 , which is not shown in FIG. 6 .", + "p0194": "The data server 2 stores, for each user, the health information received from the terminal system 1 . FIG. 7 illustrates an example of data stored in the data server 2 . As shown in FIG. 7 , the data server 2 stores, for each user, user information including the user identification information and the health information. That is, when the health information is received from the terminal system 1 of a certain user, the data server 2 additionally stores the received health information in the health information in the user information for the certain user. Thus, the health information is accumulated in the data server 2 .", + "p0195": "The data server 2 calculates secondary information at an appropriate time on the basis of the received health information . That is, the data server 2 analyzes the health information to obtain the secondary information. As described above, in the present embodiment, the secondary information is information of a statistic calculated from the received health information. For example, when average sleep hours in one week are calculated as the secondary information, the data server 2 calculates the average sleep hours each time data of sleep hours in one week is accumulated. In the present embodiment, the secondary information is also accumulated as the health information in the data server 2 .", + "p0196": "As shown in FIG. 7 , a use management table is stored in the data server 2 . The use management table will be described below.", + "p0197": "<2> Process of Providing Provision Information to Terminal System 1 by Service Server 3", + "p0198": "The terminal system 1 accesses the portal site that is managed by the portal server 41 of the service server 3 according to an instruction from a user . That is, the user performs an operation of setting the main terminal device 10 so as to be in the normal mode and accessing the portal site. There is no particular limitation on the method for logging in the portal site, and may be similar to a conventional method. In the present embodiment, a login screen is displayed on the display 16 , and the user inputs the user identification information and a password. The portal server 41 performs authentication by using the user identification information and the password. When the authentication has succeeded, the terminal system 1 is allowed to log in the portal site.", + "p0199": "In other embodiments, logging in the portal site may be performed by using personal authentication performed by the terminal system 1 . For example, when the authentication has succeeded in the sub-terminal device 20 , the main terminal device 10 is allowed to access and log in the portal site. At this time, the main terminal device 10 may not request a user to input the user identification information and the password.", + "p0200": "When the user authentication has succeeded, the portal server 41 transmits a main page of the portal site to the terminal system 1 , and the main page is displayed on the display 16 . The main page includes the provision information to be provided to the user, and the detail thereof will be described below. Therefore, when the user authentication has succeeded, the portal server 41 executes a process of generating the provision information to be provided to the user .", + "p0201": "The content of the provision information is determined on the basis of the health information, of the user, stored in the data server 2 . Therefore, the portal server 41 firstly makes a request , to the data server 2 , for use of the health information needed for determining the provision information . The health information requested in the use request is determined by using a provision condition table stored in the portal server 41 .", + "p0202": "Data stored in the portal server 41 will be described. FIG. 8 illustrates an example of data stored in the portal server 41 . As shown in FIG. 8 , the portal server 41 stores, for each user, user information, a provision condition table, and an additional provision condition table. Firstly, the user information and the provision condition table associated with the process step of step S 14 will be described, and the additional provision condition table will be described below in detail.", + "p0203": "The user information stored in the portal server 41 includes the user identification information described above, personal information, and user provision history information. The personal information is, for example, information related to an individual user, such as the name of the user, the date of the user's birth, age, and gender . The user provision history information represents the history of the provision information that has been already provided to the user. The user provision history information will be described below in detail.", + "p0204": "The provision condition table is used to determine the health information needed for generating the provision information in step S 14 . The provision condition table is preset and stored in the portal server 41 , and the content of the provision condition table may be updated at an appropriate time.", + "p0205": "FIG. 9 illustrates an example of the provision condition table. As shown in FIG. 9 , the provision condition table is a table in which provision conditions and provision contents are associated with each other. The provision condition represents a condition for providing the provision information. The provision content represents the content of the provision information to be provided when the provision condition associated therewith is satisfied. In FIG. 9 , information indicating the content itself of the provision information such as \u201cintroduction of product A\u201d is set as the provision content. However, when an identification number is assigned to each piece of the provision information prepared in the portal server 41 , information indicating the identification number may be set as the provision content.", + "p0206": "As shown in FIG. 9 , in the present embodiment, the provision condition includes a user condition and a health condition. The user condition is a condition related to a user to whom the provision information is provided. A condition related to the personal information, such as \u201cman in his 40s\u201d and \u201cmen and women in their 30s\u201d, included in the user information is set as the user condition.", + "p0207": "The health condition is a condition related to health of a user to whom the provision information is provided. As shown in FIG. 9 , the health condition includes use information indicating the health information used for determining the health condition. The use information may indicate the secondary information, such as \u201cthe average fatigue level \u201d shown in FIG. 9 , obtained by processing the health information stored in the data server 2 . Further, the use information may be information indicating the health information itself such as \u201cthe fatigue level \u201d. That is, the use request may be a request for the secondary information obtained by processing the health information, or a request for the health information itself.", + "p0208": "For example, a condition related to the fatigue index or the sleep index, such as \u201c is higher than or equal to 4\u201d or \u201c is less than or equal to five hours\u201d, is set as the health condition. The health condition may be a condition related to the health information itself , or a condition related to information obtained from the health information.", + "p0209": "The health condition is not limited to one condition related to one kind of the health information. The health condition may include a plurality of conditions related to two or more kinds of the health information. For example, the health condition may include a condition related to the fatigue index, and a condition related to the sleep index. When the information obtained by the sub-terminal device 20 is transmitted via the main terminal device 10 to the data server, the health condition may include a condition related to the information.", + "p0210": "In step S 14 described above, the portal server 41 determines the health information to be requested from the data server 2 , by using the provision condition table. That is, the portal server 41 specifies the health information used to determine the provision condition in the provision condition table, and requests the specified health information to the data server 2 . Specifically, the portal server 41 specifies the provision condition that satisfies the user condition among the provision conditions included in the provision condition table, and specifies the use information included in the provision condition. For example, in a case where the provision condition table shown in FIG. 9 is used, when a user that logs in the portal site is a man in his 40s, the use information indicating \u201cthe average fatigue level in the latest one week\u201d is specified. Therefore, the portal server 41 transmits, to the data server 2 , the use request that requests \u201cthe average fatigue level in the latest one week\u201d. When a plurality of kinds of the provision conditions satisfy the user condition, the portal server 41 specifies a plurality of pieces of the use information associated with the plurality of the provision conditions, respectively, and transmits the use request which requests the health information indicated by the specified use information.", + "p0211": "In other embodiments, sets each including the provision condition and the provision content in the provision condition table may be classified into a plurality of groups. For example, the sets may be classified into the plurality of groups according to, for example, a content of the provision information , a genre of the product/service introduced by the provision information, a company that provides the product/service introduced by the provision information, and/or a price of the product/service introduced by the provision information. At this time, the portal server 41 may cause a user to previously select a group, and may execute a process of specifying the use information merely for the provision condition included in the sets of the selected group. For example, when the sets are classified into groups according to each genre of the product/service, a genre for goods for improving sleep and a genre of supplements for recovery from fatigue, may be set. At this time, when a user selects only the genre of supplements, the portal server 41 may execute a process of specifying the use information by using the provision conditions included in the genre as subjects to be processed, in the provision condition table. Thus, the provision information that belongs to the genre selected by the user is merely provided to the user, whereby the provision information in the genre in which the user is interested can be effectively provided. A process load, on the portal server 41 , in the process of specifying the use information can be reduced.", + "p0212": "The provision condition table indicates what product/service is to be introduced according to what health state the user is in. Therefore, the content of the provision condition table may be changed on the service providing side . That is, the portal server 41 may change the content of the provision condition table according to a request from the service providing server 42 ,", + "p0213": "As described above, the use request transmitted from the portal server 41 to the data server 2 includes information indicating requested health information. FIG. 10 illustrates an example of the use request. As shown in FIG. 10 , the use request includes requested information, the user identification information, and service identification information. The requested information is information indicating the health information that is requested to be obtained, and is determined on the basis of the specified use information. In the present embodiment, the requested information includes a kind of the health information that is requested to be obtained, and a time when the health information has been obtained by the terminal system 1 .", + "p0214": "The user identification information indicates a user corresponding to the health information to be obtained. That is, the user identification information included in the use request indicates for what user the health information is to be obtained according to the use request.", + "p0215": "The service identification information is identification information assigned to the product/service provided by the service providing server. The service identification information may be set for each service provider that provides the product/service, or may be set for each kind of the product/service. The service identification information included in the use request is determined on the basis of the provision information corresponding to the health information requested by the use request. That is, it can be said that the service identification information corresponds to the provision information. In the present embodiment, the service identification information included in the use request is the service identification information assigned to the product/service introduced by the provision information corresponding to the health information requested in the use request, or the service identification information assigned to a service provider that provides the product/service.", + "p0216": "The portal server 41 previously stores a table in which the provision content included in the provision condition table is associated with the service identification information. The portal server 41 specifies the service identification information to be included in the use request with reference to the table.", + "p0217": "When the use request is received, the data server 2 performs the authentication process for the use request . The authentication process is a process of determining whether or not the health information indicated by the requested information included in the use request can be used for a service indicated by the service identification information included in the use request . In the present embodiment, the authentication process is performed by using the use management table stored in the data server 2 .", + "p0218": "FIG. 11 illustrates an example of the use management table. As shown in FIG. 11 , the use management table is a table in which the service identification information and use-allowed information are associated with each other. The use-allowed information indicates information, among the health information and the secondary information stored in the data server 2 , which can be used for the product/service indicated by the service identification information associated with the use-allowed information. The use-allowed information may indicate, for example, a kind of the health information such as \u201cthe sleep index/the fatigue index\u201d, \u201call the information\u201d, or \u201csleep index\u201d. Alternatively, the use-allowed information may indicate a period for the health information, such as \u201csleep index in the latest one month\u201d, which can be used among the health information chronologically stored. The use-allowed information may indicate, for example, a period in which the service server 3 can use the health information, such as \u201ccan be used from Dec. 1, 2013 to Dec. 1, 2014\u201d, which is not shown in FIG. 11 . That is, the use-allowed information in the use management table may indicate a condition related to a kind of the health information, a condition related to a time when the health information is obtained, or a condition related to a period in which the health information is used . In the example shown in FIG. 11 , a group in which the service identification information corresponding to product A is associated with the use-allowed information indicating that the sleep index and the fatigue index can be used, is included. By this group, it is indicated that the sleep index and the fatigue index can be used for the provision information for introducing product A.", + "p0219": "In the authentication process of step S 15 , the data server 2 determines whether or not the health information indicated by the requested information included in the received use request coincides with the health information indicated by the use-allowed information associated with the service identification information included in the use request. When both the information coincide with each other, the data server 2 determines that the authentication has succeeded, and executes process steps of steps S 16 and S 17 described below . Meanwhile, when both the information do not coincide with each other, the data server 2 determines that authentication has failed, and does not execute the process steps of steps S 16 and S 17 , which is not shown. In this case, the data server 2 transmits, to the portal server 41 , a notification that authentication has failed. In this case, the portal server 41 does not generate the provision information, and the provision information is not provided to a user in S 19 described below .", + "p0220": "When the authentication has succeeded, the data server 2 executes an analysis process corresponding to the use request . As described above, in some cases, the health information requested in the use request is information, such as \u201cthe average fatigue level in the latest one week\u201d, obtained by analyzing the health information stored in the data server 2 . In such a case, the data server 2 calculates the health information requested in the use request in a process of analyzing the health information. For example, when the average fatigue level in the latest one week is requested, an average value is calculated by using the degrees of fatigue in the latest one week among the health information, of the user, which is indicated by the user identification information included in the use request. When the health information requested in the use request is the health information itself obtained by the terminal system 1 , or is the secondary information having been already calculated in the analysis process of step S 12 described above, the analysis process may not be executed in step S 16 .", + "p0221": "Next, the data server 2 transmits, to the portal server 41 , the health information requested in the use request . Thus, the portal server 41 can obtain the health information.", + "p0222": "As described above, in the present embodiment, when the service server 3 generates the provision information, the service server 3 makes a request , to the data server 2 , for information used for generating the provision information. The data server 2 transmits, to the service server, stored information in response to the request from the service server 3 . Thus, the information obtained from the user can be managed by the data server 2 , and the information stored in the data server 2 can be used by the service server 3 .", + "p0223": "In the present embodiment, the use request includes the requested information indicating information used for generating the provision information, and identification information related to use of the information stored in the data server 2 . The data server 2 determines, on the basis of the identification information included in the use request from the service server 3 , whether or not the information indicated by the requested information included in the request is allowed to be used for generating the provision information. When it is determined that the use is allowed, the information indicated by the requested information is transmitted to the service server. Thus, the data server 2 can manage and control the use, by the service server 3 , of the accumulated information, and can perform management and control such that the kinds of the information that can be used, can be made different according to the service identification information.", + "p0224": "When the health information is received from the data server 2 , the portal server 41 generates the provision information on the basis of the received health information . That is, the portal server 41 specifies the provision information that satisfies the provision condition in the provision condition table by using the received health information. For example, when the provision condition table shown in FIG. 9 is used, the received value of the average fatigue level in the latest one week for a user who is a man in his 40s is higher than or equal to 4, the provision information for introducing product A is specified. When no provision content satisfies the provision condition, the provision information is not generated, and the provision information is not transmitted. The number of pieces of the provision information specified in the process step of step S 18 may be plural.", + "p0225": "In the process step of step S 18 , the portal server 41 stores a history of the provision information having been specified. That is, the portal server 41 updates the content of the user provision history information included in the user information for the user who has performed log-in.", + "p0226": "FIG. 12 illustrates an example of the user provision history information. In the present embodiment, the user provision history information includes information in which provided-content information, provision date information, purchase information, and additional provision time information are grouped. The provided-content information included in the user provision history information indicates the content of the provision information provided to a user, similarly to the provision content in the provision condition table described above. The provision date information indicates a time when the provision information associated therewith is provided. In the present embodiment, the provision date information indicating year, month, and day is stored. However, information indicating a time in addition to year, month, and day may be stored.", + "p0227": "The purchase information indicates whether or not the product/service introduced by the provision information associated therewith has been purchased. The additional provision time information indicates whether or not the provision information is additionally provided after the provision information associated therewith is provided, and indicates a time when the additional provision is performed in the case of the additional provision being performed. The additional provision time information will be described below.", + "p0228": "In the process step of step S 18 described above, the portal server 41 adds the information in the above-described group to the user provision history information for the specified provision information. Specifically, the portal server 41 adds, to the user provision history information, the provided-content information which indicates the provision information specified in the process step of step S 18 , the provision date information indicating year, month, and day when the provision information is provided, the purchase information indicating that purchasing is not performed, and additional provision time information indicating that additional provision is not performed. In the process step of step S 18 , the purchase information is stored so as to indicate that \u201cpurchasing is not performed\u201d and the additional provision time information is stored so as to indicate that \u201cadditional providing is not performed\u201d. However, the purchase information and the additional provision time information are updated when a product or the like is purchased, as described below.", + "p0229": "Next, the portal server 41 generates a web page including the specified provision information, and transmits the web page to the terminal system 1 . In the present embodiment, the web page is a main page which is firstly displayed when the terminal system 1 accesses the portal site. However, in another embodiment, the page including the provision information may not be the main page. When the main page is received by the terminal system 1 , the main page is displayed by the display 16 in the main terminal device 10 . Thus, the provision information is provided to the user.", + "p0230": "FIG. 13 illustrates an example of the main page displayed by the terminal system. As shown in FIG. 13 , the main page includes a provision information section 51 and a history viewing section 52 . In the provision information section 51 , the provision information is displayed. In the present embodiment, the content displayed in the provision information section 51 is defined by the provision information. In FIG. 13 , the provision information section 51 contains one kind of the provision information. However, the provision information section 51 may contain a plurality of pieces of the provision information.", + "p0231": "The provision information section 51 includes a purchase button 53 by which the product related to the provision information is purchased. That is, when a user performs an input for designating the purchase button 53 , a product purchase page for purchasing of the product is newly displayed on the display 16 , and the user is allowed to purchase the product. In the present embodiment, the product purchase page is a page provided by the portal server 41 . However, in another embodiment, the product purchase page may be a page provided by the service providing server. That is, the purchase button 53 may indicate a link to the product purchase page provided by the service providing server.", + "p0232": "The history viewing section 52 is provided so as to allow the history of the health information stored in the data server 2 to be referred to. In the present embodiment, the history viewing section 52 includes a button indicating year and month, and, when a user performs an input for designating the button, a page on which the health information for the designated year and month is displayed, is displayed on the display 16 .", + "p0233": "As described above, in the present embodiment, the service server 3 generates the provision information including information related to a product or a service for improving an index indicated by the health information of the user. Thus, useful information related to the product/service can be provided to the user. The service server 3 generates the provision information including information of the web page for allowing a user to purchase the product/service, or link information for link to the web page. Thus, when the provision information for introducing the product/service is provided, the user can purchase the product/service provided therein, in a simple operation, thereby improving usability for the user.", + "p0234": "<3> Process for Purchasing of Product", + "p0235": "Next, a process performed when a user purchases a product indicated by the provision information displayed on the main page, will be described. When a user performs an input of an operation for purchasing a product on the product purchase page, the terminal system 1 transmits, to the portal server 41 , a notification of instruction for purchase . The notification of instruction for purchase includes the user identification information and information for identifying a product to be purchased. When the notification of instruction for purchase is received, the portal server 41 transmits the notification of purchase to the service providing server 42 that provides the product to be purchased . The notification of purchase includes the user identification information and the information for identifying the product to be purchased, both of which are included in the notification of instruction for purchase.", + "p0236": "When the notification of purchase is received from the portal server 41 , the service providing server 42 executes a providing process related to providing of the product/service . For example, when data is provided as the product/service via a network, the providing process is, for example, a process of transmitting data related to the product or service. The \u201cdata related to the product or service\u201d is, for example, music data, data of recipes for meals, application data, or the like. The \u201cdata related to the product or service\u201d may be, for example, data for illumination, by the illumination section 17 of the main terminal device 10 , for inducing sleep onset or wake-up of the user. When the terminal system 1 has a loudspeaker, the \u201cdata related to the product or service\u201d may be data for outputting, from the loudspeaker, music for inducing sleep onset or wake-up of the user.", + "p0237": "When a product which is an actual object is provided, the providing process is, for example, a process of transmitting, to the terminal system 1 of the user, a notification that purchase of a product has been received, or a process of transmitting, to the terminal system 1 of the user, a notification that the product is delivered. When a service is actually provided, the providing process is a process of transmitting, to the terminal system 1 of the user, a notification that purchase of the service has been received, or a process of transmitting, to the terminal system 1 of the user, a notification of a date and time when the service is provided. For example, the service server 3 may introduce a massage service by the provision information , and perform booking of a massage on the portal site.", + "p0238": "As described above, the service server 3 may transmit, to the terminal system 1 , the data related to the product or service, according to an instruction, from a user, for purchasing the product or service being received from the terminal system 1 .", + "p0239": "When the notification of instruction for purchase is received, the portal server 41 updates the user provision history information related to the user who has issued an instruction for purchase such that purchase of the product is reflected . Specifically, the portal server 41 updates the purchase information associated with the provision information for introducing the product for which the instruction for purchase has been issued such that the content of the purchase information indicates that \u201cpurchasing has been performed\u201d. The portal server 41 updates the additional provision time information associated with the provision information such that the content of the additional provision time information indicates a time when the additional provision information is to be provided.", + "p0240": "In the present embodiment, when the product/service introduced by the provision information has been purchased, new provision information is provided after the purchase. Hereinafter, the provision information that is additionally provided anew is referred to as \u201cadditional provision information\u201d. The content of the additional provision information and the content of the provision information having been firstly provided may be different from each other, or may be the same. The content of the additional provision information is determined on the basis of the health information, of the user, which has been uploaded after the product/service has been purchased. The detail thereof will be described below.", + "p0241": "In step S 23 described above, a time when the additional provision information is provided is determined, and the additional provision time information in the user provision history information is updated such that the content of the additional provision time information indicate the determined time. The portal server 41 determines a time when the additional provision information is provided, according to the provision information having been provided. In the present embodiment, the time at which the additional provision information is provided is determined by using the additional provision condition table stored in the portal server 41 .", + "p0242": "FIG. 14 illustrates an example of the additional provision condition table. The additional provision condition table is a table for determining the content of the additional provision information and a timing for the additional provision information. As shown in FIG. 14 , the additional provision condition table is a table in which an immediately preceding provided-content, an additional provision time, an additional provision condition, and an additional provision content are associated with each other. Similarly to the provision condition table, the additional provision condition table is preset and stored in the portal server 41 , and the content of the additional provision condition table may be updated at an appropriate time. Similarly to the provision condition table, the content of the additional provision condition table may be changed on the service providing side .", + "p0243": "The immediately preceding provided-content represents content of the provision information which has been provided immediately preceding time. In other words, the immediately preceding provided-content represents the content of the provision information that causes providing of the additional provision information. The additional provision time represents a time when additional providing is performed, that is, a time when the additional provision information associated with the additional provision time is provided. In the present embodiment, the additional provision time represents, for example, a time that has elapsed from purchase of the product/service, such as \u201ctwo weeks after purchase\u201d or \u201cone month after purchase\u201d. However, in another embodiment, the additional provision time may represent: a time that has elapsed from the time when the provision information has been provided immediately preceding time; or a time that has elapsed from the time when the product has been actually used or a time that has elapsed from the time when the service has been actually performed, in the case of the time when the product has been actually used or the time when the service has been actually performed being obtained by the portal server 41 . The additional provision condition and the additional provision content will be described below in detail", + "p0244": "In step S 23 , the portal server 41 specifies the provision information for introducing a purchased product, and specifies the additional provision time associated with the immediately preceding provided-content indicating the specified provision information. The portal server 41 calculates a time when additional providing is performed, on the basis of the specified additional provision time. The additional provision time information in the user provision history information is updated so as to represent the information of the calculated year, month, and day. For example, when the current time is Dec. 6, 2013, and the specified additional provision time is \u201ctwo weeks after the purchase\u201d, information indicating Dec. 20, 2013 is stored as the additional provision time information.", + "p0245": "<4> Process of Providing Additional Provision Information after Product/Service has been Purchased.", + "p0246": "As described above, in the present embodiment, when the product/service has been purchased, a time when the additional provision information is provided is determined on the basis of the provision information for introducing the purchased product/service. The portal server 41 periodically determines whether or not the additional provision time has come, by periodically referring to the user provision history information for each user. When it is determined that the additional provision time has come, the portal server 41 executes a process of providing the user with the additional provision information for which the additional provision time has come. That is, when a predetermined time has elapsed from purchasing of the product, the above-described process is executed .", + "p0247": "Specifically, when there is the additional provision information for which provision time has come, the portal server 41 transmits, to the data server 2 , the use request for requesting use of the health information needed for determining the additional provision information . The health information needed for determining the additional provision information is the health information used for determining the additional provision condition for the additional provision information. In the process step of step S 24 , the health information needed for determining the additional provision information is determined by using the additional provision condition table stored in the portal server 41 .", + "p0248": "As shown in FIG. 14 , in the additional provision condition table, the immediately preceding provided-content, , the additional provision condition, and the additional provision content are associated with each other. The additional provision condition indicates a condition for providing the additional provision information, for the provision information indicated by the immediately preceding provided-content associated therewith.", + "p0249": "The additional provision condition includes use information indicating the health information used for determining the additional provision condition. The use information is similar to the use information included in the provision condition. However, the use information included in the additional provision condition may be set, for example, so as to designate both the health information before purchase of the product/service and the health information after purchase of the product/service, such as \u201cthe average fatigue level in the latest one week and the average fatigue level in one week immediately before the purchase\u201d.", + "p0250": "The additional provision condition represents a condition related to the health information of a user. Specifically, the additional provision condition may represent a condition related to change of the health information such as \u201cthe fatigue level has improved\u201d or \u201cthe fatigue level has not improved\u201d. The additional provision condition may be similar to the health condition described above, and, for example, the condition related to a user's state indicated by the health information, such as \u201cthe average sleep hours in the latest one week are less than or equal to five hours\u201d, may be set as the additional provision condition.", + "p0251": "The additional provision content represents the content of the additional provision information provided when the additional provision condition associated therewith is satisfied. Similarly to the provision content in the provision condition table described above, as additional provision content included in the additional provision condition table, information indicating the content itself of the provision information may be set, or information indicating the identification number assigned to the provision information may be set.", + "p0252": "In step S 24 described above, the portal server 41 firstly specifies the provision information corresponding to the additional provision information for which the provision time has come, with reference to the user provision history information. That is, the portal server 41 specifies the provision information indicated by the provided-content information associated with the additional provision time information for which the provision time has come, in the user provision history information.", + "p0253": "Next, the portal server 41 specifies the additional provision condition for the additional provision information corresponding to the provision information having been specified as described above. That is, the portal server 41 specifies the additional provision condition associated with the immediately preceding provided-content indicating the provision information having been specified, with reference to the additional provision condition table. The portal server 41 specifies the use information included in the additional provision condition having been specified, and transmits, to the data server 2 , the use request for obtaining the health information indicated by the use information having been specified. That is, the use request including the requested information indicating the health information is transmitted to the data server 2 . For example, a case where, when the additional provision condition table shown in FIG. 14 is used, the provision information indicating \u201cintroduction of product A\u201d is specified, will be described. In this case, the use information indicating \u201cthe average fatigue level in the latest one week, the average fatigue level in one week immediately before purchase\u201d, is specified and the use request for obtaining the health information indicated by the use information is transmitted to the data server 2 .", + "p0254": "The use request which is transmitted in step S 24 includes the requested information, the user identification information, and the service identification information, similarly to the use request which is transmitted in step S 14 . The contents of the three kinds of information are determined in a method similar to the method in the process step of step S 14 . For example, as in a case where the use information indicating \u201cthe average fatigue level in the latest one week, the average fatigue level in one week immediately before purchase\u201d is specified, the requested information sometimes indicates the health information in a predetermined period based on the time of purchase. In this case, the portal server 41 may use, as the purchase date, a date indicated by the provision date information of the user provision history information, or may use, when a date on which the product has been purchased is previously stored, the stored date as the purchase date, in order to calculate the predetermined period.", + "p0255": "The data server 2 executes the process similar to the process steps of step S 15 to S 17 according to the use request from the portal server 41 . That is, the data server 2 performs the authentication process for the use request . When the authentication has succeeded, an analysis process according to the use request is executed as appropriate . The health information requested in the use request is transmitted to the portal server 41 .", + "p0256": "When the health information is received from the data server 2 , the portal server 41 generates the additional provision information on the basis of the received health information . That is, the portal server 41 determines, by using the received health information, whether or not the additional provision condition specified in step S 24 described above is satisfied. The portal server 41 generates the additional provision information having additional provision content that satisfies the additional provision condition. When no additional provision content satisfies the additional provision condition, the additional provision information is not generated. The number of pieces of the additional provision information specified in the process step of step S 28 may be plural.", + "p0257": "FIG. 15 illustrates an example of a method for determining the additional provision condition. As shown in FIG. 15 , in the present embodiment, the health information obtained every day is stored in the data server 2 . As described above, in the present embodiment, whether or not the additional provision condition is satisfied is determined, in some cases, by comparison between the health information before purchase of the product/service and the health information after purchase of the product/service. In these cases, the health information in a predetermined period from the current time to a previous time is firstly analyzed as the health information of the user after the purchase, and the health information in a predetermined period from the time of the purchase to a previous time is analyzed as the health information of the user before the purchase. For example, as shown in FIG. 15 , the data server 2 calculates an average fatigue level in the latest one week and an average fatigue level in one week immediately before the purchase, as analysis results, according to the use request from the portal server 41 . The portal server 41 performs comparison between the result of the analysis of the health information before the purchase, and the result of the analysis of the health information after the purchase, whereby whether or not the health state of the user has improved is determined. For example, when the average fatigue level is reduced by a predetermined value or more, the portal server 41 determines that the health state of the user has improved, whereas when the average fatigue level is not reduced by the predetermined value or more, the portal server 41 determines that the health state of the user has not improved. The portal server 41 generates the additional provision information such that the content thereof is different according to the determination results. For example, in the example of the additional provision condition table shown in FIG. 14 , when it is determined that the fatigue level has improved, on the basis of: the average fatigue level in one week immediately before the purchase; and the average fatigue level in the latest one week, the additional provision information for introducing product D is generated. Meanwhile, when it is determined that the fatigue level has not improved, the additional provision information for introducing product E is generated.", + "p0258": "As described above, in the present embodiment, the analysis is performed on the basis of a plurality of pieces of the health information that are time-series data accumulated for a user for which the health state is analyzed. Thus, change of the health state of the user can be known. Therefore, the provision information based on such an analysis is provided, thereby providing useful information.", + "p0259": "In the present embodiment, the additional provision information according to an effect of improvement by the product or service is generated on the basis of the health information obtained by the terminal system after the product or service indicated in the provision information has been purchased. Thus, the effect of the product/service can be determined, and appropriate provision information can be provided according to the determination result.", + "p0260": "As described above, in the present embodiment, the data server 2 performs analysis by comparison between: the health state based on the information obtained by the terminal system 1 before a predetermined reference time ; and the health state based on the information obtained by the terminal system 1 after the reference time. Thus, change of the health state by the provision information can be accurately analyzed, and the provision information based on the analysis is generated, whereby the provision content can be made useful.", + "p0261": "The predetermined reference time represents, for example, a time when the product/service introduced by the provision information is purchased. However, the predetermined reference time may be any time based on the provision information having been provided to a user. For example, the reference time may be a time when the provision information is provided. For example, when a time when a user starts using the product, and/or a time when a user receives the provided service can be obtained on the server side, such a time may be used as the reference time.", + "p0262": "When the additional provision information has been generated, the portal server 41 transmits the additional provision information to the terminal system 1 . There is no particular limitation on the method for transmitting the additional provision information. In the present embodiment, a message indicating that there is the additional provision information is transmitted by push notification to the terminal system 1 . That is, when the additional provision information has been generated, the portal server 41 transmits the message in a method in which notification of the message is made even when a user does not perform an operation . That is, the terminal system 1 that receives the message, performs display of the message on the display 16 independently of the operation by the user. The user accesses the portal site by using the terminal system 1 according to the message. When the user logs in the portal site, the portal server 41 generates the main page that includes the additional provision information, and transmits the main page to the terminal system 1 . Thus, the additional provision information is provided to the user.", + "p0263": "As described above, the service server 3 transmits, to the terminal in the terminal system 1 ) of the user, the notification of the additional provision information, at a time independent of a time when the user performs the operation on the terminal device, according to the additional provision information having been generated. In other embodiments, the additional provision information itself may be transmitted to the terminal system 1 by push notification. That is, the additional provision information may be transmitted from the portal server 41 to the terminal system 1 , independently of the operation performed by the user. In other words, the additional provision information may be displayed by the terminal system 1 without access to the portal site. Thus, the notification that the additional provision information has been generated can be made to the user, at an appropriate time, by notification of the additional provision information, or by the additional provision information being transmitted to the terminal system 1 by push notification.", + "p0264": "In other embodiments, the above-described message may not be transmitted. In this case, at a time when the portal server 41 has generated the additional provision information, the user is not notified thereof, and, when the user accesses the portal site the subsequent time, the additional provision information is provided.", + "p0265": "In other embodiments, similarly to the additional provision information, the push notification may be used for the provision information . That is, the service server 3 may transmit notification of the provision information or the provision information itself, at a time independent of a time when the user performs the operation on the terminal device. More specifically, the portal server 41 may execute the process steps of steps S 14 and S 18 , according to a predetermined condition having been satisfied .", + "p0266": "As described above, in the present embodiment, after the provision information is provided, the additional provision information is provided. After the provision information is provided, the process of analyzing the information from the terminal system 1 , and the process of generating the additional provision information based on the result of the analysis may be repeatedly executed. For example, the portal server 41 may set a time that repeatedly comes, as the additional provision time in the additional provision condition table . Thus, the analysis process and the generation process are repeatedly executed each time the time comes. For example, the portal server 41 may set a group in which the provision information indicated by additional provision content is included as the immediately preceding provided-content in the additional provision condition table. For example, a group in which the provision information for introducing product D is set as the immediately preceding provided-content in the additional provision condition table shown in FIG. 14 , may be set. Thus, when a user purchases product D according to the additional provision information being provided, the additional provision information is further provided after the purchase. As described above, the analysis process and the generation process are repeated, whereby useful information can be continuously provided to a user.", + "p0268": "FIG. 16 is a flow chart showing an example of a flow of a process executed by the data server 2 . In the present embodiment, the data server 2 repeatedly executes a series of process from step S 30 to step S 37 shown in FIG. 16 . The series of process is performed by the CPU that is a processing section of the data server 2 executing a predetermined information processing program stored in a program storage section of the data server 2 . When the data server 2 includes a plurality of information processing devices, the CPUs of the respective information processing devices may share the execution of the series of process.", + "p0269": "Firstly, in step S 30 , the processing section of the data server 2 determines whether or not the health information is received from the terminal system 1 . When the result of the determination in step S 30 is affirmative, the process step of step S 31 is executed. When the result of the determination in step S 30 is negative, the process step of step S 34 describe below is executed.", + "p0270": "In step S 31 , the processing section stores the health information received from the terminal system 1 , in a storage section of the data server 2 , for each user . In step S 32 , the processing section determines whether or not the secondary information is to be calculated, on the basis of the received health information. This determination is performed according to, for example, whether or not a time when the secondary information is to be calculated has come, or whether or not the health information for calculating the secondary information has been accumulated. When the result of the determination in step S 32 is affirmative, the process step of step S 33 is performed. When the result of the determination in step S 32 is negative, the process step of step S 34 described below is executed.", + "p0271": "In step S 33 , the processing section calculates the secondary information on the basis of the received health information, and stores the calculated secondary information in the storage section of the data server 2 . As described above, the data server 2 stores the health information each time the health information is received from any one of the plurality of users, and calculates and stores the secondary information as appropriate.", + "p0272": "In step S 34 , the processing section determines whether or not the use request is received from the portal server 41 . When the result of the determination in step S 34 is affirmative, the process step of step S 35 is executed. When the result of the determination in step S 34 is negative, the process step of step S 30 is executed again.", + "p0273": "In step S 35 , the processing section executes the authentication process for the use request having been received in step S 34 , and determines whether or not the authentication has succeeded. When the result of the determination in step S 35 is affirmative, the process step of step S 36 is executed. When the result of the determination in step S 35 is negative, the process step of step S 37 is executed.", + "p0274": "In step S 36 , the processing section transmits, to the portal server 41 , the health information requested in the use request. That is, the processing section analyzes, as appropriate, the health information stored in the storage section, and transmits, to the portal server 41 , the health information, and/or the health information that is the result of the analysis .", + "p0275": "Meanwhile, in step S 37 , the processing section transmits, to the portal server 41 , notification that the authentication has failed. When the process step of step S 36 or S 37 is completed, the process step of step S 30 is executed again. In the process steps of S 30 to S 37 described above, the operation performed by the data server 2 as shown in FIG. 6 is implemented.", + "p0277": "FIG. 17 is a flow chart showing an example of a flow of a process executed by the portal server 41 . The portal server 41 repeatedly executes a series of process from step S 41 to step S 49 shown in FIG. 17 . The series of process is performed by the CPU that is a processing section of the portal server 41 executing a predetermined information processing program stored in a program storage section of the portal server 41 . When the portal server 41 includes a plurality of information processing devices, the CPUs of the respective information processing devices may share the execution of the series of process.", + "p0278": "Firstly, in step S 41 , the processing section of the portal server 41 determines whether or not access from the terminal system 1 is performed and log-in is performed by a user of the terminal system 1 . When the result of the determination in step S 41 is affirmative, the process step of step S 42 is executed. When the result of the determination in step S 41 is negative, the process step of step S 44 described below is executed.", + "p0279": "In step S 42 , the processing section generates the provision information . In step S 43 , the processing section generates the main page that includes the provision information and transmits the main page to the terminal system 1 . At this time, when the additional provision information having been already generated has not been provided yet, the processing section generates the main page including the additional provision information, in addition to the provision information. The provision information and the additional provision information are provided so as to be distinguished from each other . In the terminal system 1 , in a period from display of the main page to the log-out from the portal site by the user, the process of displaying, on the terminal system 1 , the web page is executed as appropriate according to an input from the user, which is not shown.", + "p0280": "In step S 44 , the processing section determines whether or not a product introduced by the provision information in the portal site is purchased. This determination is performed according to whether or not the notification of instruction for purchase has been received from the terminal system 1 . When the result of the determination in step S 44 is affirmative, the process step of step S 45 is executed. When the result of the determination in step S 44 is negative, the process step of step S 47 described below is executed.", + "p0281": "In step S 45 , the processing section transmits the notification of purchase to the service providing server 42 . In step S 46 , the processing section updates the user provision history information so as to reflect the purchasing of the product . As described above, at this time, a time when the additional provision information is to be provided is determined.", + "p0282": "In step S 47 , the processing section determines whether or not the time when the additional provision information is to be provided has come . When the result of the determination in step S 47 is affirmative, the process step of step S 48 is executed. When the result of the determination in step S 47 is negative, the process step of step S 41 is executed again.", + "p0283": "In step S 48 , the processing section generates the additional provision information for which the provision time has come . In step S 49 , the processing section transmits, to the terminal system 1 , the message that notifies the user of presence of the additional provision information . When the user logs in the portal site according to the message having been displayed on the terminal system 1 , the main page including the additional provision information is transmitted from the portal server 41 to the terminal system 1 by the process step of step S 43 , and the main page is displayed on the terminal system 1 . When the process step of step S 49 has been completed, the process step of step S 41 is executed again. In the process from S 41 to S 49 described above, the operation performed by the portal server 41 as shown in FIG. 6 is implemented.", + "p0285": "As described above, in the present embodiment, the terminal system 1 transmits the health information to the server system . The server system receives transmission information from the terminal system 1 , and stores the received transmission information such that the transmission information is associated with a user.", + "p0286": "The server system generates the provision information to be provided to the user for improving an index obtained from the transmission information or an index included in the transmission information , on the basis of the result of the analysis of the transmission information, and transmits the provision information to the terminal system 1 . The server system generates additional provision information that is different from the provision information having been provided, on the basis of the result of the analysis of the transmission information that is obtained by the terminal system 1 after the provision information has been provided to the user, and transmits the additional provision information to the terminal system 1 .", + "p0287": "Thus, the first provision information is provided, and a user state is analyzed on the basis of the health information having been thereafter obtained, and new provision information is generated on the basis of the result of the analysis. Thus, the second provision information can be provided in consideration of an effect exerted by the first provision information, whereby useful information can be provided to the user. For example, when the product/service introduced by the first provision information has been purchased, the effect of the product/service is determined on the basis of the health information obtained after the purchase, and the second provision information can be provided according to the effect, whereby useful information can be provided to the user.", + "p0288": "In the present embodiment, the server system stores , as the storage information, the transmission information received from the terminal system and information obtained from the transmission information such that the transmission information and the information obtained from the transmission information are associated with the user. The data server 2 receives, from the service server 3 that provides the provision information to the user of the terminal system by using the storage information, the use request that includes: the requested information indicating the information used for generating the provision information; and the service identification information for identifying a service related to the provision information, as a request for use of the storage information for generating the provision information . The data server 2 determines whether or not the information indicated by the use information included in the use request is allowed to be used for generating the provision information, on the basis of the service identification information included in the use request which is received from the service server 3 . When the use is determined to be allowed, the information that is based on the storage information and indicated by the requested information is transmitted to the service server 3 .", + "p0289": "Thus, the server system can manage and control use of the storage information by the service server. For example, the server system can also manage and control use of the storage information for each kind of use-allowed information according to the service identification information. Thus, by use of the storage information being managed and controlled, the server system can provide a service for allowing the service server to use the storage information by charging the fee according to the use. For example, in the above embodiment, a managing company that manages the data server 2 and the portal server 41 can provide a service that allows a service provider that manages the service providing server 42 to use the storage information.", + "p0290": "The server system stores association information in which the service identification information, and use-allowance information that indicates the information which is based on the storage information, and which is allowed to be used for the service indicated by the service identification information, are associated with each other. The above determination is performed on the basis of the association information. Thus, the determination is facilitated.", + "p0291": "In the present embodiment, the server system determines whether or not information that is indicated by the requested information included in the use request which is received from the service server can be used, on the basis of a condition related to the kind of the storage information, a time when the storage information is obtained, and/or a period in which the storage information is used . Thus, use of the storage information can be managed according to the kind of the storage information and/or the time when the storage information is obtained. Further, a period of use can be set for the storage information. When the above-described managing company provides a service of allowing a service provider to use the storage information as described above, the managing company can set, for example, a use fee of the storage information according to the storage information that can be used.", + "p0292": "In the present embodiment, the server system generates the provision information for introducing the product/service on the basis of the transmission information , and receives, from a user to which the provision information has been provided, a purchase instruction for purchasing of the product/service indicated by the provision information . Thus, the server system can perform not only a service of providing the provision information related to the product/service, but also a service of providing the product/service introduced by the provision information. A user is allowed to purchase the product/service indicated by the server system by making an instruction to the server system, thereby improving convenience in the purchase.", + "p0293": "In the present embodiment, the portal server 41 in the service server 3 transmits, to the service providing server 42 , a predetermined notification of purchase according to the purchase instruction having been received. The service providing server 42 executes a service providing process according to the notification of purchase having been received. Thus, the service providing process is performed by the service providing server 42 on the service provider side for providing the product/service, and purchasing of the product/service can be managed by the portal server 41 on the managing company side.", + "p0294": "In the present embodiment, the information processing system calculates an index indicating fatigue of a user. That is, the information processing system automatically obtains the biological information from a non-worn type sensor , and stores the obtained biological information . The information processing system calculates a fatigue index of the user on the basis of the stored biological information . Thus, a fatigue degree of the user can be measured without making the user conscious of the measurement.", + "p0295": "The terminal system 1 repeatedly obtains the biological information from the sensor. The biological information having been repeatedly obtained is stored and accumulated, and the fatigue index of a user is calculated on the basis of the biological information which is stored and accumulated. Thus, the fatigue degree of the user can be continuously measured.", + "p0298": "In the variation of the above embodiment, the provision condition may be automatically updated on the basis of the health information obtained from each user. In the health information obtained after the product/service introduced by the provision information has been purchased, the health state after the use of the product/service by the user is reflected. Therefore, an effect obtained by the product/service being used by the user can be inferred from the health information. In the present variation, the portal server 41 updates the provision condition on the basis of such an effect. Thus, the provision information that is more effective for the user can be provided. Hereinafter, the present variation will be described in detail.", + "p0299": "In the present variation, the portal server 41 stores a condition update table for updating of the provision condition. FIG. 18 illustrates an example of the condition update table. As shown in FIG. 18 , the condition update table is a table in which the provision content, an update condition, and an update content are associated with each other for each provision information. Similarly to the provision content in the provision condition table, the provision content represents a content of the provision information.", + "p0300": "The update condition represents a condition for updating the provision condition related to the provision information. In the present variation, the update condition includes a number-of-pieces-of-data condition, and a change condition. The number-of-pieces-of-data condition represents a condition related to the number of pieces of data for effect information indicating the effect of the product/service introduced by the provision information. The effect information represents a result of an analysis of the health information obtained after the product/service has been purchased, and represents, for example, the health information used for determining the additional provision condition. When the additional provision condition is determined, according to the product/service in a certain piece of provision information having been purchased, after the purchase thereof, the portal server 41 stores, as the effect information, the health information used for the determination. Therefore, when a plurality of users purchase the product/service, the number of pieces of effect information corresponding to the number of the users that has purchased the product/service are obtained by the portal server 41 .", + "p0301": "The number-of-pieces-of-data condition represents a condition related to the number of pieces of the effect information stored as described above. As the number-of-pieces-of-data condition, for example, a condition that \u201cthe effect information for 100 persons has been obtained\u201d is set.", + "p0302": "The change condition represents a condition related to the effect information for the number-of-pieces-of-data condition. As the change condition, for example, a condition that \u201cthe number of pieces of the effect information indicating that the effect is found is greater than or equal to a predetermined number\u201d, is set.", + "p0303": "In the present variation, the portal server 41 determines, according to the effect information having been stored for a certain piece of provision information, whether or not the provision condition for the certain piece of provision information is to be updated. Specifically, the portal server 41 specifies the number-of-pieces-of-data condition, which is associated with the provision content of the provision information in the condition update table, and determines whether or not the number of pieces of data of the stored effect information satisfies the specified number-of-pieces-of-data condition. For example, when the condition update table shown in FIG. 18 is used, the portal server 41 determines whether or not the stored number of pieces of data for the effect information reaches 100 . The result of the above determination indicates that the number-of-pieces-of-data condition is not satisfied, the portal server 41 determines that the provision condition is not updated.", + "p0304": "Meanwhile, when it is determined that the number-of-pieces-of-data condition is satisfied, the portal server 41 determines whether or not the change condition is satisfied, on the basis of the stored effect information. For example, when the condition update table shown in FIG. 18 is used, the portal server 41 determines whether or not the number of pieces of the effect information indicating that an effect is found, among the stored effect information, is greater than or equal to a predetermined number . The \u201ceffect information indicating that an effect is found\u201d is the health information for which, for example, the result of the determination as to the additional provision condition that \u201cthe fatigue level has improved\u201d shown in FIG. 14 is affirmative. That is, in the present variation, whether or not the effect information indicates that the effect is found can be determined on the basis of the result of the determination of the additional provision condition.", + "p0305": "When it is determined that the change condition is satisfied, the portal server 41 updates the provision condition table according to the update content associated with the change condition. The update content indicates the content, to be updated, of the provision condition corresponding to the provision information, for example, indicates that \u201cthe threshold value is decreased by 0.1\u201d. For example, when the provision condition that \u201c is higher than or equal to 4\u201d at the first line from the top in the provision condition table shown in FIG. 9 is updated according to the update content, shown in FIG. 18 , that \u201cthe threshold value is decreased by 0.1\u201d, the updated provision condition is a condition that \u201c is greater than or equal to 3.9\u201d. The update content may make the provision condition more lax or may make the provision condition more strict.", + "p0306": "In the above variation, an exemplary case where the provision condition table is updated is described. However, the service server 3 may update the additional provision condition table.", + "p0307": "In the above variation, the provision condition table is updated according to the change condition having been satisfied. Therefore, when the change condition is repeatedly satisfied, the updating is repeatedly performed.", + "p0308": "As described above, the server system stores, as the storage information, the transmission information received from each terminal system and/or information obtained from the received transmission information such that the transmission information and the information obtained from the transmission information are associated with a user. The server system stores a condition related to generation of the provision information to be provided to the user for improving an index obtained from the storage information or an index included in the storage information. The server system generates the provision information on the basis of the storage information and the condition. The server system analyzes the transmission information that is related to a plurality of users to which the provision information has been provided and obtained after the provision information has been provided, and updates the condition on the basis of the result of the analysis . The condition is thus updated, whereby more effective provision information can be provided.", + "p0309": "In the above variation, information obtained after the product/service introduced by the provision information has been purchased is used as \u201cinformation obtained after the provision information has been provided\u201d which is used for updating the provision condition. That is, information obtained after a time when a predetermined action due to the provision information being provided is performed, may be used as \u201cinformation obtained after the provision information has been provided\u201d. The time when the predetermined action is performed may be, for example, \u201ca time when the product/service (introduced by the provision information has been purchased\u201d or \u201ca time when the product/service (introduced by the provision information has been used\u201d, in the above variation.", + "p0310": "The condition includes at least one health condition related to the health information. Thus, the provision information according to the health information for a user can be provided. The server system updates the health condition on the basis of a result of analysis of information obtained after the provision information has been provided. Thus, the health state of the user obtained after the provision information has been provided is analyzed, and the health condition can be updated according to the result of the analysis.", + "p0311": "In the above variation, the terminal system 1 repeatedly obtains the biological information of a user, and repeatedly transmits the transmission information to the server system. The server system repeatedly receives the transmission information from each of the terminal systems 1 of a plurality of users, stores and accumulates the transmission information, and continuously updates the provision condition on the basis of the storage information that is stored and accumulated. The provision information is generated on the basis of the updated condition. FIG. 19 illustrates an effect according to the variation. The provision condition is updated as in the present variation, whereby the service server 3 can provide more effective provision information to the user. Effective provision information can be provided, whereby the number of users that use the service for providing the provision information are expected to be increased. Therefore, the data server 2 can obtain the health information for the increased number of users. When the increased number of pieces of the health information can be obtained, the provision condition can be updated so as to be more effective. Thus, the provision condition is automatically updated, whereby the cycle become advantageous, and more effective provision information can be provided.", + "p0312": "The server system determines whether or not the condition is to be updated, according to calculation of a predetermined number of the analysis results having been enabled . Thus, whether or not the condition is to be updated is determined when a predetermined number of the analysis results are obtained, whereby the condition can be accurately improved by the updating.", + "p0313": "The content of each of the provision condition table and the update condition table is not limited to the above-described content. For example, the provision condition may include a priority assigned to the provision information . When a plurality of the provision conditions are satisfied at the same time, the provision information having a higher priority may be provided . At this time, the update content included in the update condition table may be for changing the priority. For example, when the change condition indicating that an effect of improvement by the provision information is found is satisfied, the priority may be changed so as to be enhanced. Specifically, the priority level is set as \u201c5\u201d to \u201c1\u201d in order, respectively, from the highest priority level, and when the priority level for the provision information for introducing product A is \u201c3\u201d, and the change condition indicating that an effect of improvement by product A is found, is satisfied, the priority level may be changed to \u201c4\u201d in the update condition table.", + "p0314": "As described above, the provision condition may include a plurality of conditions related to the health information. At this time, by the update content included in the update condition table, the item may be added or deleted, or weighting of the item may be changed.", + "p0316": "In the above embodiment, an exemplary case is described in which, in addition to the provision condition for determining the provision information, the additional provision condition for determining the additional provision information is set. In other embodiments, the provision condition for determining the provision information may be merely set. At this time, the content of the provision condition may be automatically updated. Hereinafter, an exemplary case where the provision condition table is set and the provision condition is automatically updated, will be described as a variation of the above embodiment.", + "p0317": "FIG. 20 illustrates an example of a provision condition table according to the variation of the above embodiment. As shown in FIG. 20 , in the present variation, in the provision condition table, the provision condition, the provision content, and a subsequent provision time are associated with each other.", + "p0318": "In the present variation, the provision condition includes a plurality of conditions containing the user condition and the health condition described above. Specifically, in FIG. 20 , the provision condition includes the user condition related to the user's age, and three kinds of health conditions related to the health information . That is, the provision condition includes a condition related to an average in the total sleep hours for the latest one week, a condition related to an average in sleep efficiency for the latest one week, and a condition related to an average in the number of times of awakening during sleep for the latest one week. Depending on the provision content, all of these conditions need not be set. For example, depending on the provision content, as indicated in the provision condition associated with the provision content indicating \u201cintroduction of product C\u201d shown in FIG. 20 , some of the plurality of conditions may not be set. The conditions included in the provision condition are not limited to the above-described conditions. Another use condition and/or another health condition may be included, or another condition different from the user condition and the health conditions, may be included. Thus, in another embodiment, the provision condition may include a plurality of user conditions or may merely include one health condition.", + "p0319": "As in the above embodiment, the provision content indicates the content of the provision information that is generated when the provision condition associated therewith is satisfied.", + "p0320": "The subsequent provision time represents a time when, in a case where the provision information having the provision content associated therewith is provided, the provision information is thereafter provided . In the present embodiment, similarly to the additional provision time described above, the subsequent provision time represents a time that elapses from purchase of the product/service. However, in another embodiment, the subsequent provision time may represent a time from a time when the immediately preceding provision information has been provided. When a time when the product has been actually used or a time when the service has been actually performed, is obtained by the portal server 341 , the subsequent provision time may represent a time from this time.", + "p0321": "In the present variation, instead of the provision condition table described in the above embodiment, the provision condition table shown in FIG. 20 is used. Also in the present variation, as in the above embodiment, determination process related to the provision information is executed. In the determination process, whether or not the provision information to be provided is determined, and the provision information to be provided is determined when the provision information is provided. Also in the present variation, as in the above embodiment, when each condition included in the provision condition is satisfied, the provision information having the provision content associated with the provision condition is provided. For example, the provision condition table shown in FIG. 20 indicates that, when a user is in his/her 40s, an average in the total sleep hours for the latest one week is less than six hours, an average in sleep efficiency for the latest one week is less than 80%, and an average in the number of times of awakening during sleep for the latest one week is greater than or equal to 3, the provision information for introducing product A is determined as the information to be provided.", + "p0322": "In other embodiments, in a case where the provision condition includes a plurality of conditions, when a predetermined number of conditions among the plurality of conditions are satisfied, the portal server 41 may determine that the provision condition is satisfied. For example, in the example shown in FIG. 20 , when two conditions among three heath conditions are satisfied , it may be determined that the provision condition has been satisfied. In other embodiments, each condition included in the provision condition is subjected to weighting by the portal server 41 , and when a point of the condition that has been satisfied among the conditions is greater than or equal to a predetermined point, it may be determined that the provision condition has been satisfied.", + "p0323": "There is no particular limitation on the method for determining whether or not the provision condition is satisfied . That is, as in the above embodiment, the portal server 41 may firstly perform determination as to the user condition and then perform determination as to the health condition associated with the user condition having been satisfied. The portal server 41 may perform determination as to the user condition and determination as to the health condition individually .", + "p0324": "Also in the present variation, as in the above embodiment, when the provision condition having been satisfied is present, the provision information having the content associated therewith is transmitted to the terminal system 1 .", + "p0325": "In the present variation, when the product/service introduced by the provision information has been purchased , the portal server 41 specifies a time when the provision information is thereafter provided, on the basis of the subsequent provision time associated with the provision information. The method for specifying the time may be the same as the method for the process step for specifying the additional provision time in the above embodiment. The specified time is stored as the subsequent provision time information. That is, in the present variation, the portal server 41 stores the subsequent provision time information, instead of the \u201cadditional provision time information\u201d in the user provision history information .", + "p0326": "In the present variation, the portal server 41 periodically determines whether or not the subsequent provision time has come, by periodically referring to the user provision history information of each user. When it is determined that the subsequent provision time has come, the portal server 41 determines whether or not the provision information is to be provided and determines the provision information to be provided, by using the provision condition table. That is, when it is determined that the subsequent provision time has come, the portal server 41 executes the determination process related to the provision information. That is, in the present variation, in addition to when a user performs log-in, also when the subsequent provision time has come, the determination process related to the provision information is executed. The provision condition table used when the subsequent provision time has come, is the same as the provision condition table used when the user performs log-in, and the additional provision condition table is not used in the present variation.", + "p0327": "When, as a result of execution of the determination process related to the provision information according to the subsequent provision time having come, the provision information to be provided is generated , the portal server 41 transmits a message that the provision information is present, to the terminal system 1 , by push notification. The method for the push notification is similar to the method for step S 29 in the above embodiment. In other embodiments, the provision information itself may be transmitted to the terminal system 1 by push notification. Thus, in the present variation, when the provision information is provided according to the subsequent provision time having come, notification of the provision information is transmitted to the terminal system 1 by push notification. Thus, a notification that the provision information has been generated, can be provided to a user at an appropriate time.", + "p0328": "As in the present variation, the information processing system may not use the additional provision condition table. Even if the additional provision time is not set, when the subsequent provision time is set, the service server 3 can provide a user with the provision information at a predetermined time after the product/service has been purchased.", + "p0329": "Also in the present variation, as in the above embodiment, the provision condition is updated on the basis of the health information of the user. Also in the present variation, as in the above embodiment, the condition update table is used for updating. In the present variation, the provision condition includes a plurality of conditions . Therefore, the change condition included in the condition update table may represent conditions related to the plurality of conditions. For example, when the provision condition table shown in FIG. 20 is used, the change condition may be as follows.", + "p0330": "Among users for whom the total sleep hours or the sleep efficiency have improved after purchase of the product, a rate of users for whom the sleep efficiency has been 70% to 80% before purchase of the product is less than or equal to 30%, and a rate of users for whom the sleep efficiency has been less than or equal to 70% before purchase of the product is greater than or equal to 60%.", + "p0331": "Among users for whom the total sleep hours or the sleep efficiency have been reduced after purchase of the product, a rate of users for whom the condition of the number of times of awakening during sleep has been satisfied before purchase of the product has been greater than or equal to 80%.", + "p0332": "Among users for whom the total sleep hours or the sleep efficiency have improved after purchase of the product, a rate of users for whom WASO has been longer than or equal to one hour before purchase of the product is greater than or equal to 80%.", + "p0333": "As in the above , the change condition may include a condition related to the health information ) different form the health information included in the provision condition table.", + "p0334": "When the provision condition includes a plurality of conditions , the update content in the condition update table may be such that some of the plurality of conditions are updated. For example, when the provision condition table shown in FIG. 20 is used, the change content may be \u201cthe threshold value for the sleep efficiency is decreased by 5\u201d. For example, the change content may be associated with the change condition in the above .", + "p0335": "The change content may indicate that a new condition is added in the provision condition table. For example, when the provision condition table shown in FIG. 20 is used, the change content may indicate that a condition related to a new health information is added such as \u201cthe health condition that \u201cWASO is longer than or equal to one hour\u201d is added\u201d. For example, the change content may be associated with the change condition in the above .", + "p0336": "When each condition in the provision condition table is subjected to weighting, the change content may indicate that the weighting is changed. For example, the change content may be \u201cweighting of a condition related to the number of times of awakening during sleep is reduced.\u201d For example, the change content may be associated with the change condition in the above . As described above, when priority is assigned to the provision information, the change content may indicate that the priority is changed.", + "p0337": "The number-of-pieces-of-data condition, which is included in the update condition, may be a condition which is satisfied repeatedly when the number of pieces of data is increased. The number-of-pieces-of-data condition may be a condition that is satisfied each time a predetermined number of pieces of the effect information is obtained, for example, each time \u201cthe number of pieces of the effect information is increased by an amount corresponding to 100 persons\u201d. In this case, the service server 3 may perform determination as to the change condition on the basis of the increased predetermined number of pieces of data .", + "p0338": "As described above, the service server 3 may repeatedly determine whether or not the condition is to be updated, according to the condition related to the number of analysis results, having been satisfied. Thus, by repeated update, the content of the provision condition can be repeatedly improved. As a result, an accuracy of the provision condition can be enhanced.", + "p0339": "As described above, in the present variation, at least one of the following processes is performed as a process of updating the provision condition.", + "p0340": "Change of priority set for the provision condition", + "p0341": "Change of a parameter defined in the health condition", + "p0342": "Change of a parameter set for the health condition", + "p0343": "Addition and/or deletion of the health condition included in the provision condition", + "p0344": "The provision condition is updated according to the above to , whereby an accuracy of the provision condition can be enhanced.", + "p0346": "The service server 3 may store change information indicating the health information before and after purchase, for the purchased product service. At this time, the change information may be used as a part of the above-described effect information, or may be used for calculating the effect information. Hereinafter, an exemplary case where the change information is used will be described as the variation of the above embodiment.", + "p0347": "FIG. 21 illustrates an example of the change information. In FIG. 21 , the change information is information indicating sets in each of which the purchase information, the user information, the health information before purchase, and the health information after purchase are associated with each other. The purchase information represents information indicating the product/service purchased by a user. The purchase information is stored additionally each time the product/service is purchased. Therefore, as shown in FIG. 21 , the change information may include a plurality of kinds of the above sets for a single product. The user information represents information related to a user who has purchased the product/service. In the present variation, information representing an age and gender of a user is stored as the user information.", + "p0348": "The health information before purchase is the health information, of a user , obtained before purchase of the product. In the present variation, for the health information such as the total sleep hours, the sleep efficiency, and the sleep onset latency, information indicating an average value of the health information in one week immediately before purchase is stored as the health information before purchase. The health information stored as the health information before purchase is not limited to the above described information and may be optionally set. For example, for various kinds of the health information stored in the data server 2 , the health information before purchase may be stored.", + "p0349": "The health information after purchase represents the health information, of a user, obtained after purchase of the product. The health information after purchase is the same kind of health information as the health information, before purchase, associated therewith. That is, in the example shown in FIG. 21 , information related to the health information such as the total sleep hours, the sleep efficiency, and the sleep onset latency is stored as the health information after purchase. In the present embodiment, for the health information, information indicating an average value in one week immediately after the purchase is stored as the health information after purchase.", + "p0350": "The content of the health information before purchase and the content of the health information after purchase may represent information related to the health information of which the kind is different for each product indicated by the purchase information. For example, for product A and product B, information related to three kinds of the health information such as the total sleep hours, the sleep efficiency, and the sleep onset latency is stored as the health information before purchase and the health information after purchase. Meanwhile, for product C, information related to the total sleep hours and the fatigue level may be stored as the health information before purchase and the health information after purchase.", + "p0351": "Each of the content of the health information before purchase and the content of the health information after purchase may be different for each product indicated by the purchase information. For example, for product A and product B, an average value in one week immediately before purchase and an average value in one week immediately after purchase are stored as the health information before purchase and the health information after purchase. Meanwhile, for product C, an average value in one month immediately before purchase and an average value in one month immediately after purchase may be stored as the health information before purchase and the health information after purchase.", + "p0352": "In other embodiments, the change information may be of any form when the change information indicates change of the health state of a user. For example, the change information may include information indicating change in the health information between before purchase and after purchase, instead of the health information before purchase and the health information after purchase. The change information may include information indicating that, for example, \u201cthe total sleep hours have increased by 0.5 hours\u201d or \u201cthe sleep efficiency has improved\u201d.", + "p0353": "The change information may be stored for each provision information , or may be stored for each user information .", + "p0354": "When the product has been purchased , the portal server 41 stores the purchase information, the user information, and the health information before purchase among the change information. The health information before purchase is calculated by information used for the calculation being obtained from the data server 2 . The portal server 41 may calculate the change information not only when the product/service introduced by the provision information has been purchased but also when the product/service which is not introduced by the provision information has been purchased via the portal site 41 . Thus, the service server 3 can collect an increased amount of data for determining an effect of the product/service, whereby the effect can be accurately determined.", + "p0355": "When a predetermined period has elapsed after purchase of the product, the portal server 41 obtains and stores the health information after purchase. The predetermined period may be a previously determined period . As described above, the predetermined period may be different for each product. The predetermined period may be set as a period for the additional provision time corresponding to the product or a period for the subsequent provision time corresponding to the product. Thus, the portal server 41 can perform the process of calculating the change information and the determination process related to the additional provision information at the same timing, whereby the process can be efficiently performed. For example, in the above two processes, the process of obtaining the information from the data server 2 and the process of calculating a statistic based on the obtained information can be performed in common, whereby the process can be efficiently performed.", + "p0356": "Similarly to the calculation of the health information before purchase, the portal server 41 obtains the information used for calculation from the data server 2 , and calculates the health information after purchase by using the obtained information. When information indicating a change in the health information between before purchase and after purchase is calculated, the portal server 41 calculates the information indicating the change on the basis of the health information before purchase and the health information after purchase.", + "p0357": "The change information having been calculated as described above may be, for example, used as the effect information described above, or may be used for calculating the effect information. When the provision information is generated by using a result of the analysis of the health information related to another user , the change information may be used as the result of the analysis, or may be used for calculating the result of the analysis.", + "p0358": "As described above, the server system stores the change information in which the provision information provided to a user is associated with change, in the health information of the user, obtained before and after a predetermined reference time based on the provision information having been provided to the user. The server system updates the provision condition on the basis of the change information . Thus, the change information is stored, whereby the result of the analysis related to the health information, is easily calculated and updating can be facilitated.", + "p0360": "In other embodiments, the server system may generate the provision information on the basis of the result of the analysis related to one or more other users different from a user to whom the provision information is provided. For example, the server system may select, for example, the following user as the other users.", + "p0361": "Another user who has purchased the same product as the user to which the provision information is provided.", + "p0362": "Another user who has the user information similar to the user information of a user to whom the provision information is provided.", + "p0363": "Another user who has the health information close to the health information of a user to whom the provision information is provided.", + "p0364": "As described above, the server system may select another user on the basis of the information related to a user to whom the provision information is provided, and determine the content of the provision information on the basis of the result of the analysis related to the health information of the selected user. Thus, the useful provision information is likely to be provided to a user.", + "p0366": "In the above embodiment, the provision information is information for introducing the product/service. In the above embodiment and variation, the provision information may be any information for improving an index related to user's health and/or body. For example, the provision information may be advice information indicating advice for improving the index. The advice information may be, for example, advice for recommending an exercise to a user for whom the health information indicating that sleep is light during sleeping has been calculated, advice for recommending reconsideration of nutritional balance in meals to a user for whom the health information indicating that fatigue has been accumulated has been calculated, and advice for recommending improvement of lifestyle to a user for whom the health information indicating that the sleep hours or a bedtime are irregular has been calculated. As in the above embodiment, when the provision information is repeatedly provided, the provision information is further provided on the basis of the health information, of the user, obtained after a certain piece of provision information has been provided. Thus, since the provision information such as the advice information is repeatedly provided to a user, the user is effectively promoted to improve his/her health.", + "p0367": "When the provision information is the advice information, the service server 3 compares the health information obtained before providing of the advice information to a user with the health information obtained after providing of the advice information to the user, and may provide additional advice information as the above-described additional provision information on the basis of the result of the comparison. That is, the service server 3 may provide the advice information such that the advice information is different between when the user's health is determined to have improved and when the user's health is determined to have not improved, as a result of the comparison.", + "p0368": "The service server 3 may update the provision condition of the advice information. That is, the service server 3 compares the health information obtained before providing of the advice information to the user with the health information obtained after providing of the advice information to the user. When the service server 3 determines, as a result of the comparison, that the user's health has improved after providing of the advice information as compared to before providing thereof, the service server 3 may update the provision condition of the advice information such that the advice information is likely to be provided. Also when the provision information is the advice information, the update condition may be automatically updated by using, for example, the condition update table as in the variation described in \u201c \u201d. For example, the service server 3 may set a condition related to change in the health information of the user between before providing of the advice information and after providing of the advice information, as an update condition in the condition update table.", + "p0369": "The provision information may be data of the product/service itself for improving the index. That is, the provision information may be information for introducing the information to a user, may be advice itself to be provided to the user, or may be data itself of the product/service. The provision information may be information ) related to the product/service, or may be the advice information .", + "p0370": "In the above embodiment and variation, the provision information is for introducing the product/service, and the information processing system thus uses, as the reference time, a time at which the product/service is purchased , to perform various processes. For example, in the above embodiment, a period is designated such that a time of purchase is used as the reference time for the use information described above, or a time of the purchase is used as the reference time for the additional provision time or the subsequent provision time. In the above variation, a time of the purchase is used as the reference time for the change information . Meanwhile, when the provision information itself, such as the advice information or data of the product/service, is used, a time when the provision information is provided may be used as a reference.", + "p0372": "In the above embodiment, the health information is information that indicates an index related to user's sleep and/or fatigue. Therefore, in the present embodiment, the provision information according to user's sleep and/or fatigue can be provided. In other embodiments, the health information may be information indicating, in addition to an index related to sleep and/or fatigue, an index related to fatness , or an index related to skin condition .", + "p0374": "In the above embodiment, the service server 3 includes the portal server 41 and the service providing server 42 , and the shopping site through which the product/service is purchased, is managed by the portal server 41 . The service server 3 may have any configuration, and the functions of the portal server 41 and the service providing server 42 are not limited to the above embodiment. For example, in another embodiment, one or more service providing servers manage the shopping sites, respectively, and the provision information can be provided and the health information can be viewed on the portal site managed by the portal server. At this time, the provision information included in a page of the portal site may include link information for access to the shopping sites. When the product/service has been purchased on the shopping site, a notification of the purchase may be transmitted from the service providing server 42 to the portal server 41 in order to manage the purchasing by the portal server 41 . In other embodiments, the function of the portal server 41 and the function of the service providing server 42 may be provided by one server.", + "p0376": "In other embodiments, the product/service provided by the service server 3 may be an application to be provided to the terminal system 1 . At this time, the application may use the information stored in the data server 2 . The application described above, may be, for example, an application for performing a watching service in which the biological information obtained by the terminal system 1 is used to confirm that a user is alive and make a notification thereof to another user.", + "p0377": "In other embodiments, by the server, a game may be provided. The server that provides the game may be the same server as the service server 3 or may be a dedicated server . For example, a game application may be provided to a user on the portal site managed by the portal server 41 . The game application may be such an application as to operate on a browser for viewing of the portal site, or may be such an application as to be downloaded and installed from the server to the terminal system 1 .", + "p0378": "The game application may be obtained by the terminal system 1 in any method as well as the game application is provided from the server to the terminal system 1 . For example, a storage medium having the game application stored therein is connected to the main terminal device 10 , whereby the game application may be executed by the main terminal device 10 .", + "p0379": "The game application may be executed by the main terminal device 10 or may be executed by another user terminal of a user.", + "p0380": "As described above, when the game application is executed by a user terminal , a result of evaluation of the health information of the user may be reflected in the game. Specifically, the service server 3 evaluates the health information, of the user, stored in the data server 2 . That is, the service server 3 determines whether or not the health information satisfies a predetermined condition. The predetermined condition is, for example, a condition that a state where the fatigue level indicated by the health information is higher than or equal to a reference value and has a good value, has continued for a predetermined period, or a condition that the fatigue level has improved by a predetermine reference value or more. When the condition is satisfied, the service server 3 gives a privilege in the game to the user. For example, the service server 3 may provide the user with a content such as an item used in the game, or advance a story in the game. By giving such a privilege, the user can have a motivation for continuing calculation of the health information.", + "p0381": "In the above configuration, the information processing system that includes the terminal system and the server system executes the game process based on the game application. The information processing system refers to the health information stored in the server system when executing the game process. A predetermined process during the game process is executed on the basis of the health information having been referred to . The predetermined process is, for example, a process for providing the above-described item, or a process for advancing a story in the game. Specifically, the information processing system performs a process of adding or updating the game data used in the game application. That is, for example, the game data of an item to be provided is added or a flag in the game is updated so as to allow a new story to be played.", + "p0382": "The game process may be executed by the information processing device on the server side. Alternatively, the game process may be executed by the information processing device on the terminal side. The game process may be executed by the information processing device on the server side and the information processing device on the terminal side cooperating with each other. That is, one part of the game process may be executed on the server side, and the other part of the game process may be executed on the terminal side. For example, when the game application operates on a browser for viewing of the portal site, the game process is executed on the server side, or executed by cooperation on both the server side and the terminal side. When the game application is installed in the information processing device on the terminal side, or when the game application stored in a storage medium connected to the information processing device is executed, the game process is executed on the terminal side or executed by cooperation on both the server side and the terminal side.", + "p0383": "Adding or updating of the game data may be executed on the server side or may be executed on the terminal side. That is, the information processing device on the server side performs adding or updating of the game data stored in the storage section on the server side or the terminal side, whereby the process of adding or updating the game data may be executed. Alternatively, the information processing device on the terminal side performs adding or updating of the game data stored in the storage section on the terminal side, whereby the process of adding or updating the game data may be executed.", + "p0384": "As described above, in the game application, when the health information is used , the health information related to a user of the game application is referred to. For example, when the game application is used in a state where the user logs in the portal site, the server can specify the user by the user identification information that is inputted at the log-in. That is, the server stores the user identification information that is inputted at the log-in, and when the health information is referred to in the game process, the health information of the user specified by the stored user identification information is referred to.", + "p0385": "When the game application is used in a state where a user does not log in the portal site , the server identifies the user who uses the game application before or during the game process. Specifically, the user terminal that uses the game application receives the user identification information inputted from the user before or during the game process, and transmits the inputted user identification information to the server. The server specifies the health information to be referred to, by using the user identification information received from the user terminal. When adding or updating of the game data is executed on the server side, the server executes a predetermined process in the game process, by referring to the specified health information, on the basis of the health information having been referred to. Meanwhile, when adding or updating of the game data is executed on the terminal side, the server transmits the specified health information to the user terminal. The user terminal executes the predetermined process in the game process, by referring to the received health information , on the basis of the health information having been referred to.", + "p0386": "In the above configuration, the user identification information may double as a user's account at the portal site and an account in a service for providing the game application. In this case, the user's account is made common among a plurality of network services , whereby the user can use the plurality of network services in different user terminals in common, thereby improving usability.", + "p0387": "Second Embodiment", + "p0388": "Next, an information processing system, an information processing device, an information processing program and an information processing method according to the second embodiment will be described. The \u201c \u201d of the first embodiment described above is directed to an example in which the provision condition is updated for all the users. The second embodiment is directed to an example in which the provision condition is updated for each user. In the second embodiment, the provision condition is customized for each user, and it is therefore possible to provide information suitable for the user. The details of the second embodiment will now be described.", + "p0390": "In the configuration of the second embodiment, the information processing system includes a terminal system 100 to be described later, instead of the terminal system 1 of the first embodiment. Note however that also in the second embodiment, as in the first embodiment, the terminal system may be of any configuration. For example, the information processing system may include both of the terminal system 1 of the first embodiment and the terminal system 100 of the second embodiment. That is, the information processing system may include different terminal systems.", + "p0391": "Next, one example of the configuration of the terminal system 100 will be described. FIG. 22 shows one example of the detailed configuration of the terminal system 100 . FIG. 23 shows one example of the external appearance of the terminal system 100 . As shown in FIGS. 22 to 23 , the terminal system 100 includes the hand-held terminal 105 and the base device 106 . The hand-held terminal 105 is carried by the user. The base device 106 is placed in the house of the user, for example.", + "p0392": "In the present embodiment, the hand-held terminal 105 is a hand-held type information processing device, and the base device 106 is a cradle that is connectable to the hand-held terminal 105 . As shown in FIG. 23 , the hand-held terminal 105 can connect to the base device 106 in a detachable/reattachable manner. In addition, the base device 106 has a function of performing charging with regard to the hand-held terminal 105 , and, when the hand-held terminal 105 and the base device 106 are connected, charging of the hand-held terminal 105 by the base device 106 becomes possible. In other embodiments, a configuration in which the hand-held terminal 105 and the base device 106 are detachably/reattachably connected via a cable may be used.", + "p0393": "There is no particular limitation on the communication method between the hand-held terminal 105 and the base device 106 , and it may be wired communication via a cable or wireless communication such as radio wave communication and infrared communication. The communication between the hand-held terminal 105 and the base device 106 may be performed by a communication method in which communication is made directly therebetween or may be performed by a communication method in which communication is made via a network such as a LAN or the Internet.", + "p0394": "First, the configuration of the hand-held terminal 105 in the present embodiment will be described. The hand-held terminal 105 is a hand-held type information processing device, and, in the present embodiment, is a multifunctional device such as, for example, a mobile phone, a smart phone, or a tablet terminal. Thus, the hand-held terminal 105 has some of the various types of functions function, information processing function, network communication function, telephone call function, camera function, etc.) included in a general multifunctional device. The network communication function is a communication function realized through the Internet and/or a communication function realized through a mobile communication network. The hand-held terminal 105 may be attained by installing predetermined functions on an off-the-shelf multifunctional device. In the present embodiment, the hand-held terminal 105 is used for, in addition to being used as the multifunctional device described above, calculating the health information described above or executing the game process described above. Furthermore, the hand-held terminal 105 may be an information processing device that can be worn by the user such as, for example, a wrist watch-type or goggle-type terminal .", + "p0395": "As shown in FIG. 22 , the hand-held terminal 105 includes a communication section 110 . The communication section 110 connects to the network 4 to perform communication with the server . In the present embodiment, the communication section 110 is a communication module having the function of connecting to a mobile communication network to perform communication. For example, the communication section 110 performs communication with a communication method in compliance with telecommunications standards of 3G or telecommunications standards of 4G ). It should be noted that there is no particular limitation on the method with which the hand-held terminal 105 communicates with the server, e.g., a method with which a communication module with Wi-Fi authentication performs communication through a wireless LAN. In addition, the hand-held terminal 105 may have a function of communicating with the server through the mobile communication network and a function of performing communication with the server through the wireless LAN.", + "p0396": "The hand-held terminal 105 includes a processing section 111 . The processing section 111 executes various types of information processing to be executed by the hand-held terminal 105 . The processing section 111 is connected to the various sections 110 , and 112 to 119 of the hand-held terminal 105 . The processing section 111 has a CPU and a memory. In the hand-held terminal 105 , the various types of information processing described above are executed as a result of the CPU using the memory and executing an information processing program stored in the hand-held terminal 105 . In the present embodiment, the processing section 111 executes, as the information process, a process for calculating the health information described above, a game process, and a process for providing the user with the information received from the server, etc. When the hand-held terminal 105 operates as a multifunctional device, the processing section 111 executes information processes for achieving various functions.", + "p0397": "The hand-held terminal 105 includes an input/output interface, and functions as an information processing device for allowing the user to input and browse information. Specifically, the hand-held terminal 105 includes an operation input section 112 , a display 117 , and a speaker 118 . The operation input section 112 is an input device of any type for accepting an operation input by the user. In the present embodiment, the operation input section 112 includes buttons and a touch panel provided on the display 117 . In other embodiments, the hand-held terminal 105 may include, as the operation input section 112 , a sensor for sensing the operation of moving the hand-held terminal 105 .", + "p0398": "The display 117 , which is one example of the output device, displays various types of images generated on the hand-held terminal 105 in response to an input on the operation input section 112 , and displays various types of images based on data received from the server. The speaker 118 , which is one example of the output device, outputs various types of sounds generated by the hand-held terminal 105 in response to an input on the operation input section 112 , and outputs various types of sounds based on the data received from the server.", + "p0399": "The hand-held terminal 105 includes a sensor for sensing information for calculating the health information. In the present embodiment, the hand-held terminal 105 includes a location sensing section 113 and an environment sensor 114 .", + "p0400": "The location sensing section 113 senses the location of the hand-held terminal 105 . In the present embodiment, the location sensing section 113 senses the location by using the GNSS . The location sensing section 113 is, for example, a GPS sensor . It should be noted that there is no particular limitation on the location sensing method by the location sensing section 113 , and the location sensing section 113 may sense the location by using, for example, a beacon. Furthermore, for example, the location sensing section 113 may calculate information indicating the altitude of the user by calculating the change in altitude based on a sensing result from an atmospheric pressure sensor.", + "p0401": "The environment sensor 114 senses the environment surrounding the hand-held terminal 105 . In the present embodiment, the environment sensor 114 includes a temperature sensor and a humidity sensor. In other embodiments, an atmospheric pressure sensor, an luminous intensity sensor, a noise sensor, a smell sensor, or the like may be included in the environment sensor 114 . More specifically, the environment sensor 114 may be one that senses at least one of temperature, humidity, illumination intensity, atmospheric pressure, sound, and smell. Furthermore, in another embodiment, the microphone 115 may be used as an environment sensor for sensing noise in the surrounding area.", + "p0402": "The hand-held terminal 105 also includes the microphone 115 . The microphone 115 senses sound in the surrounding area of the hand-held terminal 105 . The microphone 115 may be used for calculating the health information. For example, the hand-held terminal 105 may sense the sound of snoring of the user by means of the microphone 115 , and calculate sleep-related information based on the sensing result. The microphone 115 may be used for accepting an audio input on the hand-held terminal 105 .", + "p0403": "The hand-held terminal 105 includes a camera 116 . The camera 116 is, for example, disposed on the same side where the display 117 is disposed on the hand-held terminal 105 . Thus, the camera 116 is disposed at a position enabling capturing an image of the user who is operating the hand-held terminal 105 . Note that the hand-held terminal 105 may determine the facial expression of the user based on the image captured by the camera 116 to calculate the fatigue level based on the facial expression of the user.", + "p0404": "The hand-held terminal 105 includes a connector 119 for forming an electrical connection with the base device 106 . In the present embodiment, when the hand-held terminal 105 is mounted on the base device 106 , the connector 119 makes contact with a connector 121 of the base device 106 . With this, communication between the hand-held terminal 105 and the base device 106 becomes possible.", + "p0405": "It should be noted that the hand-held terminal 105 includes a battery that is not diagrammatically represented, and each section of the hand-held terminal 105 operates by the power supplied from the battery. Although details will be described later, in the present embodiment, the battery of the hand-held terminal 105 can be charged by the base device 106 .", + "p0406": "Next, the configuration of the base device 106 in the present embodiment will be described. In the present embodiment, the base device 106 is disposed, for example, at the bedroom of the user, and is used for sensing biological information relating to sleep of the user while the user is in bed. Here, the biological information is information sensed from the body of the user. In the present embodiment, breathing, pulse, and body movements are sensed as the biological information. Note that in other embodiments, any information may be sensed as biological information, and any one or two of breathing, pulse and body movements may be sensed, or information other than these three information may be sensed. In addition, the base device 106 is used for providing the user in bed with a content item and information .", + "p0407": "The base device 106 includes a support section for detachably/reattachably supporting the hand-held terminal 105 . Specifically, as shown in FIG. 23 , a recessed portion in accordance with the shape of one portion of the hand-held terminal 105 is formed on a casing of the base device 106 . When the hand-held terminal 105 is inserted in this recessed portion, the hand-held terminal 105 becomes mounted on the base device 106 . Note that any mechanism may be used, with which the hand-held terminal 105 is supported on the base device 106 .", + "p0408": "As shown in FIG. 22 , the base device 106 includes the connector 121 . When the hand-held terminal 105 is inserted in the recessed portion, the connector 119 of the hand-held terminal 105 and a connector 121 of the base device 106 are connected. As a result, communication between the hand-held terminal 105 and the base device 106 becomes possible, and charging of the hand-held terminal 105 by the base device 106 becomes possible.", + "p0409": "The base device 106 includes a Doppler sensor 124 which is one example of the sensor for sensing the biological information. The Doppler sensor 124 , by discharging microwaves and receiving reflected waves of the discharged microwaves, senses a moving object based on a difference between the frequency of the discharged microwaves and the frequency of the received microwaves. In the present embodiment, the Doppler sensor 124 emits radio waves in the forward direction of the base device 106 . In the present embodiment, the subject to be sensed by the Doppler sensor 124 is the user, and body movements of the user are sensed by the Doppler sensor 124 . Although details will be described later, analysis such as frequency analysis performed on the sensed biological information allows further calculation of biological information other than body movements such as breathing and pulse.", + "p0410": "The base device 106 includes a power obtaining section 123 for obtaining power from an external power supply. In the present embodiment, the base device 106 is connected to a power plug and an AC adapter via a power cord that is not diagrammatically represented. When the power plug is connected to an electrical outlet which is an external power supply, power is supplied to the power obtaining section 123 of the base device 106 . The base device 106 operates by the power from the external power supply obtained by the power obtaining section 123 . In addition, the power obtaining section 123 performs charging of the hand-held terminal 105 by transmitting the supplied power to the hand-held terminal 105 through the connector 121 . In other embodiments, the base device 106 may include a battery, and power charged in the battery may be transmitted to the hand-held terminal 105 . Furthermore, in the present embodiment, although charging is performed in a mode in which power is supplied through the connector, in other embodiments, power may be supplied through non-contact charging.", + "p0411": "The base device 106 includes a projector 125 for projecting an image on a screen or a wall surface . The projector 125 may be any display device that displays an image on a surface away from the base device 106 by projecting the image on the surface. In the present embodiment, as shown in FIG. 23 , a projector 125 is formed on the base device 106 such that a light projection section 125 a faces upward, i.e., such that the image is projected upward. More specifically, in the present embodiment, the projector 125 projects the image on the ceiling. In the present embodiment, for example, the projector 125 may display an image that induces the user to fall asleep or wake up , and displays an image showing an evaluation result of sleep when the user awakens in the morning.", + "p0412": "In the present embodiment, the base device 106 corrects the image to be projected on the ceiling by using, if necessary, a technology of so-called projection mapping. More specifically, the base device 106 corrects the image such that an image in accordance with the unevenness and/or the color of the projection plane of the projector 125 is displayed. Note that conventional methods may be used as the method for correcting the image. The base device 106 includes a camera 127 for correcting the image. As shown in FIG. 23 , the camera 127 is formed on the base device 106 in a direction that includes an image capturing range of the location where the image is to be projected by the projector 125 . Thus, the camera 127 is provided so as to face the same direction as the projector 125 .", + "p0413": "The base device 106 includes a speaker 126 . The speaker 126 is used for, for example, outputting a sound that induces the user to fall asleep or wake up .", + "p0414": "The base device 106 includes a control section 122 that controls the various sections 123 to 127 of the base device 106 . The control section 122 is connected to each of the sections 121 and 123 to 127 of the base device 106 . The control section 122 executes various types of control processes executed by the base device 106 . The control section 122 has a memory and a CPU. In the base device 106 , the various types of control processes are executed when the CPU uses the memory and executes information processing programs stored in the base device 106 . For example, a control section 122 controls charging operation of the hand-held terminal 105 by controlling the power obtaining section 123 . In addition, the control section 122 causes the projector 125 and/or the speaker 126 to reproduce information and a content item to be provided to the user on the base device. Furthermore, the control section 122 transmits information sensed by the Doppler sensor 124 to the hand-held terminal 105 .", + "p0415": "It should be noted that the base device 106 may include other elements in addition to or instead of those shown in FIG. 22 . For example, the base device 106 may include an environment sensor, a display, a nondirectional speaker, a light source , and/or a smell generation device, etc. Note that when the base device 106 includes an environment sensor, the hand-held terminal 105 may not include an environment sensor. The hand-held terminal 105 and the base device 106 may include environment sensors of the same type or may include environment sensors of different types.", + "p0416": "Note that the configuration of the data server 2 and the service server 3 of the second embodiment is similar to that of the first embodiment. Therefore, also in the second embodiment, as in the first embodiment, the terminal system 100 transmits the health information to the server, and the server generates the provision information based on the health information to transmit the provision information to the terminal system 100 . Note however that in the second embodiment, the terminal system 100 itself also executes the process of deciding the provision information to be provided to the user. Therefore, in the second embodiment, the server does not need to execute the process of transmitting the provision information to the terminal system 100 .", + "p0419": "Next, the outline of the operation of the terminal system 100 according to the second embodiment will be described. The following description is directed to an example in which a tune to be reproduced as a sleep-inducing content item is provided as the provision information to be provided to the user. Note that a sleep-inducing content item is a piece of music and/or an image that induces the user to fall asleep. That is, in the present embodiment, a tune for inducing the user to fall asleep is reproduced when the user gets in bed. The terminal system 100 first presents, to the user, information representing candidate tunes to be reproduced , and reproduces a tune selected by the user from among the candidate tunes. Therefore, in the present embodiment, the information of candidate tunes can be said to be the provision information, or a tune that is actually reproduced can be said to be the provision information.", + "p0420": "The terminal system 100 has a decision rule for deciding candidate tunes pre-stored, and decides candidate tunes in accordance with the decision rule. Note that the decision rule is a condition for deciding information of a candidate tune , which is the provision information, and it can therefore be said that the decision rule is a provision condition described above. In the present embodiment, the decision rule is updated for each user. That is, in the present embodiment, the provision condition is set for each user, and is customized for each user. The outline of the flow of the process of the terminal system 100 will now be described with reference to FIG. 24 .", + "p0421": "FIG. 24 illustrates an example flow of the process performed by the terminal system 100 according to the second embodiment. FIG. 24 shows the process to be executed by the terminal system 100 during the period from when the user gets in bed until after the user wakes up.", + "p0422": "Also in the second embodiment, as in the first embodiment, the terminal system 100 starts measuring the biological information when the user gets in bed . Specifically, the user docks the hand-held terminal 105 onto the base device 106 when getting in bed. As the hand-held terminal 105 is docked onto the base device 106 , the terminal system 100 determines that the user has gotten in bed . In response to sensing the user getting in bed, the terminal system 100 starts the measurement for calculating the health information of the user.", + "p0423": "The method for calculating the health information in the second embodiment is similar to that of the first embodiment, and the health information including the sleep indices and the fatigue index is calculated. Note that it is assumed in the second embodiment that the hand-held terminal 105 has the functions of the various sections shown in FIG. 4 . That is, in the second embodiment, the biological information is transmitted from the base device 106 to the hand-held terminal 105 , and the hand-held terminal 105 calculates the health information based on the biological information. Note that in other embodiments, the functions of the various sections shown in FIG. 4 may be provided in the hand-held terminal 105 or may be provided in the base device 106 .", + "p0424": "Note that there is no particular limitation on the condition for starting the measurement. For example, in other embodiments, in which the hand-held terminal 105 and the base device 106 capable of wireless communication with each other, the base device 106 may start the measurement in response to an instruction for starting the measurement being transmitted from the hand-held terminal 105 to the base device 106 . The base device 106 may determine, at intervals of a predetermined amount of time, whether or not the user is around the base device 106 by using the Doppler sensor 124 , for example, and start the measurement in response to determining that the user is around.", + "p0425": "Next, the terminal system 100 executes the process of providing the user with the information representing candidate tunes described above . In the present embodiment, the hand-held terminal 105 decides candidate tunes based on the health information that has been calculated so far . The hand-held terminal 105 identifies the tendency of sleep for the past one week to decide candidate tunes in accordance with the tendency, the details of which will be described later. For example, when it is identified that the user tends to have trouble falling asleep , tunes that are thought to be effective in helping the user fall asleep are decided as candidate tunes. The hand-held terminal 105 provides the user with information representing the decided candidate tunes as the provision information. Specifically, information representing the decided candidate tunes is displayed on the display 117 of the hand-held terminal 105 .", + "p0426": "FIG. 25 illustrates an example image to be displayed on the hand-held terminal 105 for providing information representing candidate tunes. As shown in FIG. 25 , a candidate tune image 131 representing the decided candidate tunes is displayed on the display 117 of the hand-held terminal 105 . Note that the present embodiment is directed to an example in which three tunes are presented to the user as candidate tunes, but there may be any number of candidate tunes. In the present embodiment, an advice image 132 representing advice information for the user is displayed, together with the candidate tune image 131 . The advice image 132 will be described later.", + "p0427": "As the tune information image 131 is displayed on the display 117 , the user selects a tune of interest from among the candidate tune . The selection of a tune is done, for example, by the operation of touching on the tune of interest from among the candidate tunes displayed on the display 117 . In response to this operation, the hand-held terminal 105 starts reproducing the selected tune . Specifically, the hand-held terminal 105 transmits the sound data of the tune to the base device 106 , and the base device 106 , having received the sound data, outputs the tune from the speaker 126 . Note that in other embodiments, the tune may be reproduced from a speaker 118 of the hand-held terminal 105 , in addition to from the speaker 126 of the base device 106 .", + "p0428": "Note that a list of tunes that can be reproduced by the hand-held terminal 105 may be stored in the hand-held terminal 105 or may be stored in the server. When the list is stored in the server, the hand-held terminal 105 obtains the list from the server at an appropriate point in time . The sound data of the tunes included in the list may be stored in the hand-held terminal 105 or may be stored in the server. When the sound data of the tunes is stored in the server, the hand-held terminal 105 obtains the sound data of selected tunes from the server at a point in time when the tune to be reproduced is selected, for example.", + "p0429": "In the present embodiment, in a period of time during which the user is asleep , the terminal system 100 determines the sleep state of the user based on the biological information during this period, and controls the operation of the hand-held terminal 105 in accordance with the sleep state. Specifically, the hand-held terminal 105 determines whether or not the user has fallen asleep based on the obtained biological information. Then, if it is determined that the user has fallen asleep, the hand-held terminal 105 stops the reproduction of the tune, which was started in step S 104 described above . This is because it is not so needed to reproduce the tune because the user has fallen asleep. Note that in other embodiments, the hand-held terminal 105 may stop the reproduction of the tune after the passage of a predetermined amount of time from the point in time when the user falls asleep, or may stop the reproduction of the tune at a point in time when it is determined that the user is in a deep sleep . In other embodiments, the tune does not need to be stopped depending on the sleep state of the user. For example, in other embodiments, the reproduction of a tune may be stopped after the passage of a predetermined amount of time from the start of the reproduction of the tune.", + "p0430": "In other embodiments, the terminal system 100 may control operations other than controlling the reproduction of a tune. For example, in other embodiments, the terminal system 100 may perform a control of changing the state of the hand-held terminal 105 depending on the sleep state of the user.", + "p0431": "When the user wakes up, the terminal system 100 calculates the health information for the current sleep period . As described above, the terminal system 100 calculates the health information based on the biological information obtained in the current sleep period by a method similar to the first embodiment. Note that it is possible to determine that the user has woken up based on the biological information.", + "p0432": "Next, the hand-held terminal 105 generates a question for the user, and presents the generated question to the user . The question is generated based on the health information calculated in step S 106 , the details of which will be described later. For example, if it is determined, from the calculated health information, that the number of mid-sleep awakenings is greater than normal , the hand-held terminal 105 asks a question such as \u201cDid you sleep well?\u201d. If the fatigue level of the user is higher than normal, for example, the hand-held terminal 105 asks a question such as \u201cDid you recover from fatigue?\u201d.", + "p0433": "FIG. 26 illustrates an example image to be displayed on the hand-held terminal 105 at the time of awakening. As shown in FIG. 26 , when the user wakes up, an evaluation result image 135 and a question image 136 are displayed on the display 117 .", + "p0434": "The evaluation result image 135 represents the evaluation result relating to sleep and fatigue for the current sleep period. Specifically, the evaluation result image 135 represents the sleep quality level and the energy level, which are the evaluation result calculated based on the biological information . The sleep quality level is an index representing how good the sleep is . While there is no particular limitation on the method for calculating the sleep quality level, the sleep quality level is calculated based on the sleep indices described above so that a higher numerical value represents a better state, for example. The energy level is a numerical value calculated for the purpose of representing the user's health state in an easy-to-understand manner for the user. The energy level is calculated based on the fatigue level described above, for example, and a higher numerical value represents a better state . The energy level may be calculated by any method, and the energy level may be calculated by, for example, subtracting the fatigue level from 100. Thus, in the present embodiment, the sleep quality level and the energy level are calculated as the evaluation result relating to the health information. Note that the sleep quality level and the energy level as described above are also types of health information. In the present embodiment, the evaluation result relating to the health information is presented to the user when the user wakes up, and the user can therefore know the evaluation result immediately after waking up.", + "p0435": "The question image 136 represents the question generated in step S 107 . In FIG. 26 , the question image 136 representing the question \u201cDid you sleep well?\u201d is generated and displayed. In the present embodiment, the question image 136 includes button images used by the user to answer. Note that the answer of the user may be given in any form, e.g., a form in which the user selects an answer from among two or more choices, or a form in which the user inputs a numerical value.", + "p0436": "When the question image 136 is displayed on the display 117 , the user inputs an answer to the question . In the present embodiment, the user gives an answer by making an input of specifying a button image included in the question image 136 .", + "p0437": "In response to the answer being input, the hand-held terminal 105 updates the decision rule . The decision rule is updated based on the health information calculated in step S 106 and the answer of the user input in step S 108 , the details of which will be described later. For example, when the health information indicates that the health state is good or when the user input is an answer indicating that the health state is good, it can be determined that the tune, as a sleep-inducing content item, reproduced when falling asleep was effective for the user. Therefore, in such a case, the decision rule is updated so that this tune will more likely be selected. On the other hand, when the health information indicates a poor state or when the user input is an answer indicating that the health state is poor, it can be determined that the tune, as a sleep-inducing content item, reproduced when falling asleep was not effective for the user. Therefore, in such a case, the decision rule is updated so that this tune will less likely be selected.", + "p0438": "As described above, in the second embodiment, the terminal system 100 can provide the user with information of candidate tunes that are decided based on the health information . Thus, the terminal system 100 can recommend suitable tunes, as sleep-inducing content items, to the user based on the health information.", + "p0439": "Moreover, in the second embodiment, the terminal system 100 determines the effectiveness of the tune provided when the user awakens out of sleep . The decision rule for deciding the tune is updated based on the determination result. Since the decision rule is updated based on the user health information of each individual user and the answer of each individual user, the decision rule becomes customized for each user. Thus, the terminal system 100 can provide provision information suitable for each user, and it is possible to provide information that is useful for each individual user.", + "p0441": "Next, a specific example of the tune provision process to be executed on the terminal system 100 will be described. As described above, the tune provision process is a process of providing a tune to be reproduced when the user falls asleep.", + "p0442": "FIG. 27 is a functional block diagram showing an example functional configuration of the terminal system 100 . In the present embodiment, as elements for executing the tune provision process, the terminal system 100 includes an analysis section 151 , a health information storage section 152 , a genre decision rule storage section 153 , and a tune decision rule storage section 154 . In the present embodiment, the various sections 151 to 154 are implemented by the processing section 111 and/or the storage section of the hand-held terminal 105 . Specifically, the analysis section 151 is implemented by the processing section 111 executing a predetermined information processing program. The various storage sections 152 to 154 are implemented by the predetermined storage section described above. The predetermined storage section is any storage medium that can be accessed by the processing section first portion 1 , and it may be a storage medium built in the hand-held terminal 105 or may be an external storage medium that can be attached to/detached from the hand-held terminal 105 .", + "p0443": "The analysis section 151 performs an analysis relating to the health of a user based on the biological information of the user, and provides the user with information based on the analysis result . As shown in FIG. 27 , the analysis section 151 includes a health information calculation section 161 , a state identification section 162 , a genre decision section 163 , a tune decision section 164 , and an information provision section 165 .", + "p0444": "The health information calculation section 161 obtains the biological information sensed by the base device 106 , and calculates the health information based on the obtained biological information. The method for calculating the health information is similar to that of the first embodiment. The calculated health information is output to the health information storage section 152 .", + "p0445": "The health information storage section 152 stores the biological information sensed by the base device 106 and the health information calculated by the health information calculation section 161 . The health information storage section 152 stores the latest biological information and health information and those obtained in the past over a predetermined storage period . Therefore, when the biological information and the health information are obtained, the health information storage section 152 updates the information so that information for the storage period is stored. Note that in the present embodiment, the health information storage section 152 does not need to store all types of sensed biological information and calculated health information, but may store those of the sensed biological information and the calculated health information that are used for deciding candidate tunes.", + "p0446": "Based on the health information of a user, the state identification section 162 identifies the health state of the user. In the present embodiment, as the health state of the user, the state identification section 162 calculates the sleep tendency for a predetermined period of time . Specifically, the state identification section 162 obtains the health information from the health information storage section 152 , and calculates at least the average sleep latency and the average number of mid-sleep awakenings for the past one week based on the obtained health information. Note that there is no particular limitation on the information to be calculated as the health state. The state identification section 162 may calculate a state relating to a sleep index other than the sleep latency and the number of mid-sleep awakenings, or may calculate a state relating to fatigue, for example. The information to be calculated as the health state may be calculated based on the latest health information of the user and past health information as in the present embodiment, or may be calculated based on the latest health information of the user in other embodiments. The information calculated as the health state may be calculated based on health information of the user and health information of other users. For example, in other embodiments, the state identification section 162 may calculate, as the health state, information based on a comparison between the health information of the user and health information of other users . Note that the terminal system 100 may obtain the health information of other users from the server at an appropriate point in time.", + "p0447": "The genre decision section 163 decides a genre of candidate tunes to be provided to the user based on the health state identified by the state identification section 162 . In the present embodiment, tunes that can be reproduced by the hand-held terminal 105 described above are each assigned a genre. That is, in the list of tunes that can be reproduced, the identification information of each tune is associated with information representing the genre of the tune. The genre decision section 163 decides a predetermined number of genres from among a plurality of different genres assigned to different tunes.", + "p0448": "The decision of a genre is made by using a genre decision rule. The genre decision rule storage section 153 stores the genre decision rule. FIG. 28 illustrates an example genre decision rule. In the present embodiment, as shown in FIG. 28 , the genre decision rule is information that associates the condition relating to sleep tendency with the weight set for each genre. Note that although not shown in the figure, the genre decision rule includes information that associates each genre with a reference weight value predetermined for that genre.", + "p0449": "The genre decision section 163 determines, for each sleep condition, whether the identified health information satisfies the sleep condition included in the genre decision rule, and identifies the setting associated with the satisfied sleep condition. The genre decision section 163 changes the weight value assigned to each genre in accordance with the identified setting. For example, in the example shown in FIG. 28 , if the identified health information indicates that the sleep latency is 15 minutes or less and that the number of mid-sleep awakenings is one to three, there are identified a setting of increasing the weight value for genre A by 0.5 and a setting of increasing the weight value for genre D by 0.5. As a result, the weight values for genre A and genre D are each increased by 0.5.", + "p0450": "As described above, in the present embodiment, changes are made to the predetermined reference weight values by using the genre decision rule. The genre decision section 163 decides the genre based on the weight values to which changes have been made. There is no particular limitation on the method for deciding the genre based on the weight values. For example, the genre decision section 163 may select a genre for which the weight value is largest. For example, the genre decision section 163 may decide the genre by using a random number so that each genre is selected with a probability based on the weight value. Then, randomness can be introduced to the genre decision, and it is possible to increase the possibility that many different genres are selected. That is, the hand-held terminal 105 can present tunes of many different genres to the user.", + "p0451": "The tune decision section 164 decides candidate tunes from among tunes that are included in the genre decided by the genre decision section 163 . The candidate tunes are decided by using the tune decision rule. The tune decision rule storage section 154 stores the tune decision rule. The tune decision rule includes a table in which each of the tunes that can be reproduced by the hand-held terminal 105 is associated with the weight assigned to the tune. FIG. 29 illustrates an example table included in the tune decision rule. In the present embodiment, as shown in FIG. 29 , each tune is associated with the weight value in the table. Each tune is associated with a genre.", + "p0452": "The tune decision section 164 selects candidate tunes from among the various tunes included in the decided genre based on the weight values assigned to the tunes. There is no particular limitation on the method for selecting candidate tunes based on the weight value. For example, the tune decision section 164 may select, as candidate tunes, a predetermined number of tunes, of all the tunes included in the decided genre in the descending order of the weight value. For example, the tune decision section 164 may select a predetermined number tunes by using a random number so that each tune is selected in accordance with its weight value from among the various tunes included in the decided genre. Then, randomness can be introduced to the tune decision, and it is possible to increase the possibility that many different tunes are selected. That is, the hand-held terminal 105 can present many different tunes to the user.", + "p0453": "The information provision section 165 provides, to the user, information of each of the candidate tunes decided by the tune decision section 164 . Specifically, the tune decision section 164 displays the candidate tune image 131 on the display 117 . Note that in the present embodiment, the candidate tune image 131 includes recommendation information such as \u201crecommended for those who have trouble falling asleep\u201d, for example, in addition to the title of the candidate tune . The advice image 132 representing advice information is displayed, in addition to the candidate tune image 131 . The content of these recommendation information and advice information is determined based on the health state identified by the state identification section 162 , for example. For example, a table in which the condition relating to the health state and the content of the recommendation information and advice information to be presented is pre-stored in the hand-held terminal 105 , and the content of these information is identified using this table. That is, the hand-held terminal 105 identifies the content of the recommendation information and advice information associated with a condition, of all the conditions included in the table, that is satisfied by the health state. The content of information thus identified is displayed on the display 117 as the recommendation information and the advice information.", + "p0454": "Moreover, the information provision section 165 accepts a user input of selecting one of the candidate tunes, and decides the tune selected by the user as a tune to be reproduced as a sleep-inducing content item. Then, the information provision section 165 obtains the sound data of the decided tune, and reproduces the tune by using the speaker 118 . Thus, the tune is presented to the user as a sleep-inducing content item.", + "p0456": "Next, referring to FIG. 27 , a specific example of the decision rule updating process to be executed on the terminal system 100 will be described. The decision rule updating process is a process of determining the effectiveness of the tune as a sleep-inducing content item and updating the decision rule depending on the determination result. In the present embodiment, as elements for executing the decision rule updating process, the terminal system 100 includes the health information calculation section 161 described above, the question presenting section 155 , and the update section 156 . In the present embodiment, the sections 155 and 156 are implemented by the processing section 111 of the hand-held terminal 105 executing a predetermined information processing program.", + "p0457": "As described above, when the user awakens, the health information calculation section 161 calculates the health information based on the biological information obtained during the current sleep period .", + "p0458": "The question presenting section 155 presents a question for the user based on the health information calculated by the health information calculation section 161 . In the present embodiment, based on the health information, the question presenting section 155 selects one of a plurality of questions that are prepared in advance. While there is no particular limitation on the method for deciding the question to be presented, the present embodiment uses a table in which conditions relating to the health information are associated with questions.", + "p0459": "FIG. 30 illustrates an example table used for deciding a question to be presented to the user. As shown in FIG. 30 , the table associates each of the conditions relating to the health information with a question to be presented when the condition is satisfied. The question presenting section 155 identifies a condition that is satisfied by the calculated health information from among the conditions included in the table, and selects a question that is associated with the identified condition. In the example shown in FIG. 30 , if there is a difference of one or more between the number of mid-sleep awakenings for the current sleep period and the average value , for example, a question \u201cDid you sleep well?\u201d is selected.", + "p0460": "Note that the method for deciding a question may be a method based on the latest health information and the past health information for the user as in the present embodiment, or may be a method based on the latest health information for the user in other embodiments. The question may be decided based on the health information of the user and the health information of other users. For example, in other embodiments, the question presenting section 155 may decide a question based on information that is based on a comparison between the health information of the user and information obtained from health information of other users .", + "p0461": "The question presenting section 155 generates the question image 136 representing the selected question and displays the question image 136 on the display 117 . Note that when there are a plurality of questions selected, only a predetermined number of questions thereof may be presented to the user, or all of the selected questions may be presented. As a question is presented as described above, the user inputs an answer to the question .", + "p0462": "The update section 156 obtains the answer input by the user for the question. The update section 156 also obtains the health information calculated by the health information calculation section 161 . The update section 156 updates the tune decision rule, of all the decision rules, based on the health information and the answer of the user. In the present embodiment, the tune decision rule is updated by using a rule update table, which is an example of the condition update table described above. The rule update table is stored in the tune decision rule storage section 154 , for example.", + "p0463": "FIG. 31 illustrates an example rule update table used for deciding the update content. As shown in FIG. 31 , the rule update table is for associating each condition relating to the health information or the user's answer with the content of an update to be implemented when the condition is satisfied. The update section 156 identifies a condition that is satisfied by the health information or the user's answer, of all the conditions included in the rule update table, and identifies the content of the update that is associated with the identified condition. In the example shown in FIG. 31 , if the sleep time is greater than or equal to the average value , for example, update content indicating an increase of the weight value by 0.1 is identified. Note that if there are a plurality of conditions satisfied, the update section 156 identifies update content that is associated with each of the plurality of conditions.", + "p0464": "Note that the method for deciding the update content may be a method based on the latest health information and the past health information for the user as in the present embodiment, or may be a method based on the latest health information for the user in other embodiments. The update content may be decided based on the health information of the user and the health information of other users. For example, in other embodiments, the update section 156 may decide the update content based on information that is based on a comparison between the health information of the user and information obtained from the health information of other users .", + "p0465": "The update section 156 updates the tune decision rule stored in the tune decision rule storage section 154 in accordance with the identified update content. Specifically, in the table included in the tune decision rule described above , the weight value associated with a tune to be updated is increased or decreased in accordance with the identified update content. Note that the \u201ctune to be updated\u201d is a tune that is reproduced as a sleep-inducing content item in the current sleep period.", + "p0466": "Note that in the rule update table, a condition indicating that the health information is good or a condition such that the answer of the user indicates that the health state is good is associated with such update content that the weight value is increased . In the rule update table, a condition indicating that the health information is poor or a condition such that the answer of the user indicates that the health state is poor is associated with such update content that the weight value is decreased . Therefore, in the present embodiment, when it is estimated that the tune reproduced as a sleep-inducing content item is effective in improving the sleep of the user, the tune will more likely be provided as a candidate tune in the future. On the other hand, when it is estimated that the tune reproduced as a sleep-inducing content item is not effective in improving the sleep of the user, the tune will less likely be provided as a candidate tune in the future.", + "p0467": "Note that in other embodiments, the update of the decision rule may be done based on the health information calculated in the past, in addition to the health information for the current sleep period . For example, based on new health information and past health information, the hand-held terminal 105 may determine whether or not the health state of the user has improved and decide the update content in accordance with the determination result. Specifically, when it is determined that the health state of the user has improved, the hand-held terminal 105 may increase the weight value of the tune to be updated, and when it is determined that the health state of the user has not improved, the hand-held terminal 105 may decrease the weight value of the tune to be updated.", + "p0468": "While the update section 156 updates the decision rule based on the calculated health information and the answer of the user in the present embodiment, it may update the decision rule based on one of the health information and the answer of the user in other embodiments. In other embodiments, the update section 156 may update the genre decision rule together with the tune decision rule.", + "p0469": "There is no particular limitation on the method for updating the decision rule. In other embodiments, the update section 156 may update the condition included in the decision rule instead of changing the update content included in the decision rule. For example, the update section 156 may change the threshold value included in the condition . For example, consider a situation in which the condition \u201cthe sleep time is 6 hours or less\u201d is set as a sleep condition in the sleep condition described above for determining if the user is having sufficient sleep. In this situation, if the calculated health information indicates that the health state is good even though the actual sleep time is 6 hours or less, the update section 156 may change \u201c6 hours or less\u201d, which is the threshold value of the condition that \u201cthe sleep time is 6 hours or less\u201d, to \u201c5 hours 50 minutes\u201d, for example.", + "p0470": "As described above, the present embodiment is directed to an example using a table shown in FIG. 28 to FIG. 31 as a means for deciding the content of the provision information or updating the provision rule. There is no particular limitation on the means for deciding the content of the provision information or updating the provision rule. For example, the content of the provision information may be decided by an analysis engine. The analysis engine uses a predetermined algorithm to analyze sleep/fatigue based on the health information, evaluate characteristic portions based on the analysis, estimate the cause of each characteristic portion, and provide a solution to the cause. Then, the update of the provision rule may be done by updating the program of the analysis engine.", + "p0472": "Next, a specific example of the process to be executed on the terminal system according to the second embodiment will be described. FIG. 32 is a flow chart showing an example flow of a process to be executed on the hand-held terminal 105 . In the present embodiment, the series of processes shown in FIG. 32 is started in response to sensing the user getting in bed, i.e., in response to the hand-held terminal 105 being docked onto the base device 106 . Note that in other embodiments, the series of processes may be started in response to the start of communication between the hand-held terminal 105 and the base device 106 or in response to the user performing a predetermined start operation on the hand-held terminal 105 or the base device 106 .", + "p0473": "First, in step S 120 , the hand-held terminal 105 executes a start-up process. The start-up process is a process that is executed in response to the start of the series of processes shown in FIG. 32 . Specifically, the terminal system 100 starts sensing by the sensor for sensing the biological information for calculating the health information . That is, the processing section 111 of the hand-held terminal 105 gives the base device 106 an instruction to start the sensing operation. In response to this instruction, the control section 122 of the base device 106 causes the Doppler sensor 124 to start the sensing operation.", + "p0474": "When the start-up process of step S 120 is executed , the base device 106 executes a charge start process. In the charge start process, the base device 106 starts charging the hand-held terminal 105 . Specifically, the control section 122 gives a power obtaining section 123 an instruction to start the charge. In response to this instruction, the power obtaining section 123 supplies power from an external power source to the hand-held terminal 105 via the connector 121 . Note that it is assumed that the base device 106 is connected to an external power source . Note that the base device 106 may check the remaining battery level of the hand-held terminal 105 , and may start the charging operation on the condition that the remaining battery level is less than or equal to a predetermined amount. The charging operation is ended in response to the battery of the hand-held terminal 105 is fully charged.", + "p0475": "In step S 121 , the hand-held terminal 105 executes the tune provision process described above . This starts the reproduction of the tune as a sleep-inducing content item.", + "p0476": "After the process of step S 121 , the process of steps S 122 to S 125 to be described later is executed repeatedly during the sleep period of the user. Note that in the present embodiment, the process loop of steps S 122 to S 125 is executed at the rate of once per a predetermined amount of time.", + "p0477": "In step S 122 , the hand-held terminal 105 obtains the sensing result of the Doppler sensor 124 . The Doppler sensor 124 , which has started the sensing operation by the process of step S 120 described above, outputs the sensing result to the control section 122 . The control section 122 transmits the sensing result to the hand-held terminal 105 . Thus, the sensing result of the Doppler sensor 124 is obtained by the hand-held terminal 105 . Note that the control section 122 may transmit the information of the sensing result of the Doppler sensor 124 as it is to the hand-held terminal 105 , or may process the sensing result before transmitting it to the hand-held terminal 105 .", + "p0478": "In step S 123 , the hand-held terminal 105 calculates the sleep information . That is, the processing section 111 calculates the various sleep indices based on the sensing result obtained in step S 122 . The calculation of the sleep indices is done by a method similar to that of the first embodiment. Note that in step S 123 , the processing section 111 may calculate the sleep information used for determining the sleep state of the user in steps S 124 and S 125 to be described later. In step S 123 , the processing section 111 may not calculate the sleep indices , the fatigue information and the energy level, which can only be calculated at the end of the sleep period.", + "p0479": "In step S 124 , the hand-held terminal 105 executes an information process in accordance with the sleep state of the user. That is, the processing section 111 determines whether or not the sleep state of the user has become a predetermined state. Then, when it is determined that it has become a predetermined state, the operation of the hand-held terminal 105 , the operation mode and/or the settings are controlled. In the present embodiment, when it is determined that the user has fallen asleep, the processing section 111 stops the reproduction of the tune, which was started in step S 121 described above .", + "p0480": "In step S 125 , the hand-held terminal 105 determines whether or not the user has awakened. That is, the processing section 111 determines whether or not the user has awakened based on the biological information obtained in step S 122 and/or the sleep indices calculated in step S 123 . When it is determined that the user has awakened, the series of processes of steps S 126 to S 129 is executed. On the other hand, if it is determined that the user has not awakened, the process of step S 122 is executed again. That is, the series of processes of steps S 122 to S 125 is executed repeatedly until it is determined that the user has awakened.", + "p0481": "In step S 126 , the hand-held terminal 105 calculates the health information based on the information obtained during the sleep period . The health information is calculated by a method similar to that of the first embodiment.", + "p0482": "In step S 127 , the hand-held terminal 105 presents a question to the user based on the health information calculated in step S 126 , and accepts an input of an answer to the question by the user . That is, the processing section 111 displays the question image 136 described above on the display 117 , and obtains the input of the answer from the operation input section 112 .", + "p0483": "In step S 128 , the hand-held terminal 105 updates the decision rule based on the health information calculated in step S 126 and the answer input by the user in step S 127 .", + "p0484": "In step S 129 , the hand-held terminal 105 transmits the health information calculated in step S 126 to the server. That is, the processing section 111 transmits the calculated health information to the server by means of the communication section 110 . Thus, the health information for one sleep period is transmitted to the server and stored in the server. Thus, in the present embodiment, the hand-held terminal 105 automatically transmits, to the server, information to be transmitted. That is, the information is uploaded to the server even without an instruction from the user.", + "p0485": "After step S 129 described above, the hand-held terminal 105 ends the series of processes shown in FIG. 32 . The processing section 111 gives an instruction to stop the sensing operation to the base device 106 . In response to this instruction, the control section 122 of the base device 106 stops the sensing operation of the Doppler sensor 124 .", + "p0486": "Note that in the present embodiment, if the hand-held terminal 105 is taken off the base device 106 for some reason during the sleep period, the base device 106 cannot transmit the sensing result of the Doppler sensor 124 to the hand-held terminal 105 . Then, the base device 106 stores, in its storage section , the data of the sensing result that has not been transmitted to the hand-held terminal 105 . Then, in response to the hand-held terminal 105 being next docked onto the base device 106 , the base device 106 transmits the data of the sensing result stored in the storage section to the hand-held terminal 105 . The hand-held terminal 105 , having received the data, calculates the sleep indices based on the sensing result . Note that the hand-held terminal 105 may not execute a control process based on the calculated sleep indices . This is because the calculated sleep indices are based on a past sensing result.", + "p0487": "If it is determined that the user has awakened based on the calculated sleep indices , the hand-held terminal 105 executes the processes of steps S 126 to S 129 described above. Thus, even if the hand-held terminal 105 is taken off the base device 106 while the user is asleep, the decision rule is updated and the health information is transmitted to the server when the hand-held terminal 105 is next docked onto the base device 106 . Therefore, when the user awakens and notices that the hand-held terminal 105 is off the base device 106 , for example, the user can dock the hand-held terminal 105 onto the base device 106 . Then, the terminal system 100 can update the decision rule and transmit the health information to the server.", + "p0488": "Note that in other embodiments, when the hand-held terminal 105 and the base device 106 are capable of wireless communication with each other, the processes of steps S 122 to S 125 described above can be executed continuously even if the hand-held terminal 105 is taken off the base device 106 .", + "p0489": "Note that the hand-held terminal 105 may execute a process similar to the process to be executed by the main terminal device 10 in the normal mode of the first embodiment, in addition to the process shown in FIG. 32 . That is, the hand-held terminal 105 may access the service server 3 to display the health information of the user on the display 16 and display the provision information provided from the service server 3 on the display 16 .", + "p0491": "In the second embodiment, the decision rule, which is an update condition, is updated based on the personal health information of the user of the terminal system 100 . In other embodiments, the decision rule may be updated based on health information of a plurality of users. In the second embodiment, the decision rule is updated for the user of the hand-held terminal 105 . That is, the decision rule is one that is used only for the user himself/herself of the terminal system 100 . In other embodiments, the decision rule may include a part that is common among a plurality of users. As a variation of the second embodiment, an example will now be described below in which a part of the decision rule is used commonly among a plurality of users and updated based on health information of a plurality of users.", + "p0492": "FIG. 33 is a functional block diagram showing an example functional configuration of the terminal system 100 according to a variation of the second embodiment. In FIG. 33 , as elements for executing the decision rule updating process, the terminal system 100 includes the health information calculation section 161 described above, the question presenting section 155 , a first update section 171 , and a second update section 172 . Note that those elements in FIG. 33 that are denoted by the same reference signs as those in FIG. 32 are the same elements as those of FIG. 32 , and they will not be described below in detail.", + "p0493": "The first update section 171 is the same as the update section 156 in the second embodiment. In this variation, the terminal system 100 includes the second update section 172 in addition to the first update section 171 . In this variation, the sections 171 and 172 are implemented by the processing section 111 of the hand-held terminal 105 executing a predetermined information processing program.", + "p0494": "The second update section 172 updates the genre decision rule, of all the decision rules. In this variation, the genre decision rule is updated based on health information of a plurality of users. Specifically, the server has a genre decision rule stored therein as a provision condition, and also stores the health information of each user received from his/her terminal system, as in the first embodiment. In addition to the health information, the server receives information representing a tune as a sleep-inducing content item from the terminal system and stores the information in association with the health information. At a predetermined point in time, the server updates the genre decision rule based on health information relating to a plurality of users.", + "p0495": "There is no particular limitation on the predetermined point in time, and it may be a point in time that is determined by the administrator of the server , for example. For example, the predetermined point in time may be one for which the data count condition and/or change condition set in the server, as described in \u201c \u201d above. For example, the server may determine that the predetermined point in time is when a predetermined number of health information associated with tunes of a certain genre have been obtained.", + "p0496": "There is no particular limitation on the content of update of the genre decision rule. In this variation, based on health information that is associated with tunes of a certain genre, the server determines the effectiveness of the tunes of that genre and updates the genre decision rule based on the effectiveness. For example, with an example genre decision rule shown in FIG. 28 , when based on the health information associated with tunes of genre C, it is determined that the tunes are effective in shortening and improving the sleep latency, the server increases the increment for the weight value of genre C. As described above, the server may calculate information representing whether or not the health state has improved, based on the history of health information , as the effectiveness of tunes as sleep-inducing content items.", + "p0497": "When the genre decision rule is updated, server transmits, to the terminal systems, an update notification including information representing the update content. Each of the terminal systems updates the genre decision rule stored therein in accordance with the update notification. That is, when the update notification from the server is received by the terminal system 100 , the second update section 172 updates the genre decision rule stored in the genre decision rule storage section 153 in accordance with the update content represented by the update notification. Thus, the content of the genre decision rule stored in the server can be synchronized with the content of the genre decision rule stored in the terminal system 100 .", + "p0498": "Note that the server may store genre decision rules that are applied to all the users of the network service, or may store genre decision rules each applied to some of the users of the network service that belong to a predetermined group. For example, depending on sex, age, address and/or hobby and preferences, etc., the users are divided into predetermined groups. The server stores a genre decision rule corresponding to each group, and when the genre decision rule is updated, the server transmits the update notification to terminal systems of the users belonging to the group. Thus, the update notification is not transmitted to terminal systems of the users not belonging to the group.", + "p0500": "In the variation described above, the tune decision rule is updated for each user based on health information of the user, and the genre decision rule is updated based on health information of a plurality of users. In other embodiments, the terminal system 100 may update the rules by any method that is based both on health information of each user and health information of a plurality of users. For example, in other embodiments, rules may be updated by the following method.", + "p0501": "FIG. 34 is a functional block diagram illustrating an example functional configuration of a terminal system according to another variation of the second embodiment. Note that FIG. 34 shows only some of the functional elements of the terminal system, and the same elements as those of FIG. 27 or FIG. 33 are omitted. As shown in FIG. 34 , in this variation, the analysis section 151 includes a first tune decision section 181 and a second tune decision section 183 , instead of the genre decision section 163 and the tune decision section 164 . The terminal system 100 includes a first tune decision rule storage section 182 and a second tune decision rule storage section 184 , instead of the tune decision rule storage section 154 and the genre decision rule storage section 153 .", + "p0502": "In this variation, the first tune decision section 181 calculates the weight value of each tune that can be reproduced by the hand-held terminal 105 , based on the health state identified by the state identification section 162 . The weight value of the first tune decision section 181 is determined by using the first rule stored in the first tune decision rule storage section 182 . There is no particular limitation on the method for deciding the weight value. For example, the weight value may be decided by using information that is obtained by changing a setting of the genre decision rule shown in FIG. 28 to a setting relating to the weight assigned to a tune. That is, a first rule may be information that associates a condition relating to the sleep tendency with a setting relating to the weight assigned to a tune. Then, the first tune decision section 181 determined, for each sleep condition included in the first rule, whether or not the health information identified by the state identification section 162 satisfies the sleep condition, and updates the weight value assigned to a tune in accordance with the setting associated with the satisfied sleep condition. Note that in this variation, the first rule is updated based on health information of a plurality of users , the details of which will be described later.", + "p0503": "The second tune decision section 183 correct the weight value of each tune calculated by the first tune decision section 181 based on the health state identified by the state identification section 162 . The correction by the second tune decision section 183 is done by using a second rule stored in the second tune decision rule storage section 184 . There is no particular limitation on the specific method of correction. For example, as a second rule, similar to the first rule, a correction may be done by using information that associates a condition relating to the sleep tendency and the correction to the weight assigned to a tune. Then, for each sleep condition, the second tune decision section 183 determines whether or not the health information identified by the state identification section 162 satisfies the sleep condition included in the second rule, and corrects the weight value assigned to a tune in accordance with a setting that is associated with the satisfied sleep condition. Note that in this variation, the second rule is updated based on the health information of each user , the details of which will be described later.", + "p0504": "Note that the information of the health state used by the first tune decision section 181 for calculating the weight value may be the same as, or different from, the information of the health state used by the second tune decision section 183 for correcting the weight value. For example, the first tune decision section 181 may use the health state identified by the state identification section 162 based on the latest health information of the user, while the second tune decision section 183 uses the health state identified by the state identification section 162 based on past health information of the user. For example, the first tune decision section 181 may use the health state of the first type , while the second tune decision section 183 uses the health state of the second type .", + "p0505": "The second tune decision section 183 selects candidate tunes as described above based on the corrected weight value. Then, the second tune decision section 183 outputs information representing the selected candidate tunes to the information provision section 165 . Note that the method for selecting candidate tunes based on weight values may be similar to that of the second embodiment described above.", + "p0506": "As described above, in this variation, the decision rules include the first rule and the second rule. In this variation, the first rule is updated based on health information of a plurality of users, and the second rule is updated based on health information of each user . Thus, also in this variation, as in the second embodiment described above, it is possible to update the provision condition for each user and to customize the provision condition for each user.", + "p0507": "While there is no particular limitation on the method for updating the first rule, the first rule may be updated by a method similar to that for updating the genre decision rule described in the \u201c \u201d above, for example. That is, although the first rule and the genre decision rule are different from each other in whether weight values are assigned to tunes or genres of tunes, a similar method can be used as the method for updating the weight value. For example, when it is determined that a certain tune is effective in improving sleep, the server updates the first rule so that the weight value of the tune is likely set to a great value. Then, the server transmits, to each terminal system, an update notification including information representing update content relating to the first rule. Each terminal system updates the first rule stored therein in accordance with the update notification.", + "p0508": "While there is no particular limitation on the method for updating the second rule, the second rule may be updated by a method similar to that for updating the tune decision rule described in the embodiment above, for example. That is, the terminal system 100 may decide update content of the second rule in accordance with the rule update table at a point in time in response to awakening of the user. For example, when it is estimated that a tune that has been reproduced as a sleep-inducing content item is effective in improving the sleep of the user, the terminal system 100 updates the second rule so that the tune is more likely to be provided as a candidate tune in the future.", + "p0509": "According to the variation described above, a result based on the first rule applied to a plurality of users is corrected based on the second rule, which is customized for each user, to decide the content of the provision information based on the corrected result. Then, the second rule allows the terminal system 100 to provide information that reflects the health state of each user, and it is possible to make a proposal suitable for each user, as in the second embodiment described above.", + "p0510": "Note that in the variation described above, the terminal system 100 can decide a candidate tune without using the second rule by using the weight value calculated based on the first rule. Therefore, in other embodiments, the terminal system 100 may decide a candidate tune using the weight value calculated based on the first rule, without using the second rule, under a predetermined condition. For example, the terminal system 100 may determine whether or not to use the second rule in accordance with a instruction from the user. For example, in the initial period of use of the terminal system 100 , when the precision of the second rule is expected to be low , the second rule may not be used.", + "p0512": "In other embodiments, the terminal system 100 may provide provision information using the environment information sensed by the environment sensor 114 . Specifically, the health information storage section 152 obtains and stores environment information during the sleep period of the user. The state identification section 162 calculates the tendency of sleep taking the environment information into consideration. Then, the genre decision section 163 and/or the tune decision section 164 decide the candidate tunes based on the tendency of sleep taking the environment information into consideration. For example, when the air temperature is used as the environment information, the state identification section 162 calculate the relationship between the sleep latency and the air temperature for the user of the terminal system 100 . Then, if the user tends to have trouble falling asleep when the air temperature is low, and if it is determined that the air temperature is low based on the environment information obtained when the provision information is provided , the genre decision section 163 may change the weight value so that a genre that is thought to be effective in helping the user fall asleep is more likely to be selected.", + "p0514": "In other embodiments, the terminal system 100 may update the decision rule by using the environment information sensed by the environment sensor 114 . Specifically, the health information storage section 152 obtains and stores environment information during the sleep period of the user. The update section 156 decides update content of a decision rule between the health information calculated based on the biological information obtained during a sleep period and the environment information obtained during the sleep period. For example, when the weight value of a tune to be updated is changed based on the number of mid-sleep awakenings indicated by the health information, the update section 156 may decide the amount by which the weight value is changed based on the air temperature during the sleep period. That is, when the air temperature is too high or too low during the sleep period, the number of mid-sleep awakenings is believed to increase due to the air temperature. Therefore, in such a case, even if the number of mid-sleep awakenings is large, the update section 156 sets a relatively small amount as the amount by which the weight value is changed . On the other hand, when the air temperature is adequate during the sleep period , the number of mid-sleep awakenings is not believed to increase due to the air temperature. Therefore, in such a case, if the number of mid-sleep awakenings is large, the update section 156 sets a relatively large amount as the amount by which the weight value is changed .", + "p0516": "In other embodiments, the terminal system 100 may provide provision information and/or update a decision rule based on the location information sensed by the location sensing section 113 . For example, the terminal system 100 may calculate activity information indicating the activities of the user for one day based on the location information. The activity information is information that is calculated from the history of location information for one day, and may be information representing the amount by which the user traveled or information representing the activities of the user , for example. The terminal system 100 decides the content of the provision information based on the activity information. The terminal system 100 may calculate the fatigue level of the user from the activity information and select different genres depending on the fatigue level, for example.", + "p0517": "The terminal system 100 may decide update content of a decision rule based on the activity information. For example, the terminal system 100 may calculate the fatigue level of the user from the activity information for one day, and may vary the update content depending on the fatigue level when updating the decision rule when the user wakes up the next day.", + "p0519": "The second embodiment is directed to an example in which the provision information is a tune to be reproduced as a sleep-inducing content item and a tune to be a candidate of that tune. There is no particular limitation on the content of the provision information, and the provision condition can be updated for any provision information for each user. For example, the provision information may be recommendation information introducing a product, or the like, to the user as in the first embodiment, and/or advice information for improving the health state of the user.", + "p0520": "For example, when the hand-held terminal 105 can provide a plurality of different functions , the hand-held terminal 105 may provide a recommended function, from among the plurality of different functions, as the provision information to the user. Specifically, the hand-held terminal 105 may introduce the function of reproducing a sleep-inducing content item to a user who has been determined to have trouble falling asleep from the obtained health information, and may introduce the function of reproducing an awakening content item before awakening, to a user who has been determined to have trouble waking up.", + "p0522": "In other embodiments, the base device 106 may communicate with the server without using the hand-held terminal 105 therebetween. For example, the base device 106 may have the function of communicating with the server via the network 4 , as does the hand-held terminal 105 . Then, the base device 106 may transmit information sensed by the base device 106 directly to the server or may transmit the information both to the hand-held terminal 105 and to the server. The base device 106 may receive the provision information directly from the server. Then, the server may transmit the provision information both to the hand-held terminal 105 and to the base device 106 or may transmit the provision information to either one of them.", + "p0524": "In the second embodiment and the variation thereof, the information processing system performs an analysis relating to the health of a user based on the biological information of the user, and provides information based on the analysis result to the user . The information processing system updates, for each user, based on the biological information of the user, at least a part of a rule for deciding the information to be provided based on the biological information . The information processing system decides the information to be provided to the user, by using a rule that is set for each user as a result of the update . Thus, it is possible to update the rule to one of content that is suitable for the user as an individual, and it is therefore possible to provide information that is useful for each user.", + "p0525": "Updating a rule means to include the following processes, for example.", + "p0526": "Update of information processing program for deciding information to be provided", + "p0527": "Update of table describing rules for deciding information to be provided", + "p0528": "Update of algorithm for deciding information to be provided", + "p0529": "Update of analysis engine for deciding information to be provided", + "p0530": "Note that the biological information for updating a rule may be biological information obtained at any point in time. The information processing system may update a rule based on biological information that is obtained after the provision of the provision information, which is decided based on the rule before being updated. Then, a rule can be updated so as to reflect the effectiveness of the provision information based on the rule before being updated.", + "p0531": "In the second embodiment and the variation thereof, the rule includes a first part to be updated for each user , and a second part that is common among a plurality of users including the present user . In other words, as the rule for deciding the information to be provided, the information processing system uses both the first rule common among a plurality of users and the second rule which is set individually for each of the plurality of users . Then, the first part makes it possible to provide information suitable for each user, while the rule can be made a common rule for those parts that are common among a plurality of users, thereby simplifying the rule updating process.", + "p0532": "Note that the first part and the second part may be used for deciding the same provision information as in variations described in \u201c and \u201c \u201d above. The first part and the second part may be those that are used for deciding information of the same type , as are the first rule and the second rule described above. For example, the first part and the second part may be used for deciding different provision information. That is, the content of the first provision information may be decided based on the first part, the content of the second provision information may be decided based on the second part, and the first and second provision information may be provided to the user .", + "p0533": "Note that in a variation of the second embodiment, the information processing system calculates the first information used for deciding the provision information by using the biological information and the first rule. Moreover, the information processing system decides the provision information by using the calculated first information and the second rule. Then, it is possible to decide the provision information by processing a result based on the first rule which is common among the users in accordance with individual second rule of the user.", + "p0534": "Moreover, in the variation described above, the information processing system calculates the second information by using the second rule and the biological information, and decides the provision information based on the first information and the second information . Then, when the first information is a type of information with which the provision information can be decided without using the second rule, the terminal system 100 can operate both in the mode in which the provision information is decided by using the first information without using the second rule and in the mode in which the provision information is decided by using the second rule.", + "p0535": "In a variation of the second embodiment, the information processing system obtains biological information for a plurality of users . The information processing system updates the second part based on at least a plurality of biological information included in the biological information . Then, the second part which is common among a plurality of users can be updated to suitable content taking into consideration the health state of the plurality of users. Note that \u201cupdate based on biological information\u201d means to include updating based on health information that is calculated based on biological information.", + "p0536": "Note that as in the variation described above, the condition for updating the first part may be different from the condition for updating the second part. The frequency with which the first part is updated may be higher than the frequency with which the second part is updated. Since the first part is updated based on health information of an individual user, the first part can be updated to suitable content for the user by updating it with a high frequency. In contrast, since the second part is updated based on health information of a plurality of users, the second part can be updated to suitable content for a plurality of users by updating it based on a certain number of health information.", + "p0537": "In a variation of the second embodiment, the information processing system includes one or more user terminal and a server system that is capable of communicating with the user terminal via a network. The user terminal has the function of analyzing the health of the user and updating the rule. Each user terminal updates the first part based on the biological information obtained by the user terminal. The server system obtains biological information of a plurality of users. The second part is updated based on a plurality of biological information obtained by the server. Then, it is possible to easily update the rule by obtaining a plurality of biological information by the server.", + "p0538": "In the second embodiment and the variation thereof, the information processing system updates at least a part of the rule for each user based on the biological information of the user and an input by the user . Then, it is possible to update the rule taking into consideration the subjective element of the user . Note that there is no particular limitation on the input by the user, and it is not limited to the input of an answer to a question as in the embodiment described above. For example, the information processing system may perform an update based on an input that is voluntarily made by the user . The information processing system presents a question to the user , and uses the answer to the question as an input by the user. This makes it easy for the user to make an input, and allows the information processing system to obtain useful user inputs.", + "p0539": "Moreover, in the second embodiment and the variation thereof, the information processing system decides the content of a question based on the biological information of the user that is obtained after the provision of the information, and presents the question to the user. Then, the information processing system updates the rule based on the biological information of the user that is obtained after the provision of the information and the answer to the question. Then, the information processing system can present an appropriate question to the user, and obtain an input of an answer that is useful for updating the rule.", + "p0540": "In a variation of the second embodiment, the information processing system obtains environment information relating to the environment around the user when sensing the biological information . The information processing system decides the information to be provided based on the biological information of the user and the environment information relating to the user. Then, it is possible to provide the user with information that is more useful by taking into consideration the environment around the user. The information processing system updates at least a part of the rule based on the biological information of the user and the environment information relating to the user. Then, it is possible to more appropriately update the rule by taking into consideration the environment around the user.", + "p0541": "In a variation of the second embodiment, the information processing system presents to the user, as the provision information, information relating to the health of the user and/or information for improving the health of the user. A tune or a candidate tune as a sleep-inducing content item described above can be said to be \u201cinformation relating to the health of the user\u201d or \u201cinformation for improving the health of the user\u201d. The advice information and the recommendation information in the first embodiment or information of the function for improving the health as described in \u201c \u201d above can also be said to be \u201cinformation relating to the health of the user\u201d or \u201cinformation for improving the health of the user\u201d.", + "p0542": "In a variation of the second embodiment, the information processing system repeatedly executes an analysis based on the biological information that is obtained repeatedly . The information processing system stores, in a predetermined storage section , at least a part of the biological information obtained repeatedly and the information calculated in the repeatedly-executed analysis. The information processing system updates, for each user, at least a part of the rule based on the biological information for a plurality of iterations and/or the information calculated over a plurality of iterations of the analysis , which are stored in the storage section. Then, the rule is updated by using the results of a plurality of measurements and/or analyses, and it is therefore possible to provide to the user information that is more useful. The information processing system repeatedly updates the rule, and it is therefore possible to gradually change the rule to one of content that is suitable for the user. Moreover, the information processing system updates the rule each time the analysis is performed. Then, since the rule is updated each time the analysis is performed, it is possible to update rule frequently.", + "p0543": "<Other Variations>", + "p0545": "The process distribution between the terminal side and the server side, the process distribution between server devices and the process distribution between devices within the terminal system described above are illustrative, and there is no particular limitation on the manner in which the various processes on the information processing system are distributed. For example, while the decision rule is stored on the terminal side in the second embodiment, a part or the whole of the decision rule may be stored on the server side in other embodiments. Then, the process of deciding the content of the provision information and the process of updating the decision rule may be executed on the server side.", + "p0546": "As described above, the embodiment described above can be used as an information processing system, an information processing server, and the like, with the aim of providing useful information, etc.", + "p0547": "While certain example systems, methods, devices and apparatuses have been described herein, it is to be understood that the appended claims are not to be limited to the systems, methods, devices and apparatuses disclosed, but on the contrary, are intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.", + "features": { + "F1": "An information processing system comprises at least one processor and a memory storing computer readable instructions.", + "F2": "The information processing system uses a rule set for each user to decide content to be reproduced for aiding sleep of the user.", + "F3": "The information processing system reproduces content decided based on the rule.", + "F4": "The information processing system performs an analysis relating to the health of the user based on biological information detected under an environment where the content is being reproduced.", + "F5": "The information processing system updates the rule to increase the likelihood of content reproduction when the analysis indicates an improved health state of the user.", + "F6": "The rule for selecting content includes a first rule for deciding specific content to be reproduced and a second rule for deciding a specific genre of content.", + "F7": "The second rule is updated based on biological information of a plurality of users.", + "F8": "The first rule is updated for each user based on their biological information and/or the second rule.", + "F9": "The system can obtain biological information of each of the plurality of users.", + "F10": "A server system configured to communicate with user terminals over a network.", + "F11": "User terminals are configured to execute analysis and rule updates.", + "F12": "The processor is configured to update at least a part of the rule based on biological information of the user and user input.", + "F13": "The processor presents a question to the user and uses their answer as input.", + "F14": "The processor decides content of the question based on biological information.", + "F15": "The system obtains environment information relating to an environment around the user during biological information sensing.", + "F16": "The processor provides the user with information for improving health.", + "F17": "The storage medium stores biological information and analysis information for repeated use.", + "F18": "The processor repeatedly updates the rule based on the results of consecutive analyses.", + "F19": "The biological information of the user can be obtained from sensors detecting pulse, breathing, or body movements.", + "F20": "The content can be a composition or movie for aiding sleep.", + "F21": "The content can be specific genre songs.", + "F22": "The analysis result indicates improved health if the user falls asleep quickly after content reproduction.", + "F23": "A weight value for the content is increased if the analysis is positive.", + "F24": "Improved health state is determined based on multiple sleep periods.", + "F25": "The rule is updated based on detected health states of group-associated users.", + "F26": "The group of users is determined based on demographic information.", + "F27": "A weight is assigned to specific content based on the first rule and corrected by the second rule.", + "F28": "Weights assigned to content can be updated using the second rule and can be selected using the first rule without the second rule in accordance with user instruction.", + "F29": "The first rule is updated based on the user's sleep time, while the second rule updates based on sleep latency of users.", + "F30": "An information provision method comprises similar processes to the system functions described." + } + } + }, + { + "FAN": "1864211", + "CPC": [ + "G06F-016/00", + "G06F-016/24", + "G06F-016/24544", + "G06F-016/2468", + "G06F-016/285", + "G06F-016/90335", + "G06F-017/30/386", + "G06F-017/30/466", + "G06F-017/30/542", + "G06F-017/30/598", + "G06F-017/30/979", + "G06F-017/40", + "G06N-007/02", + "G06N-007/02/3", + "G06N-020/00", + "G06N-099/00/5", + "Y10S-706/90" + ], + "Content": { + "title": "Fuzzy data operations", + "pa01": "A method for clustering data elements stored in a data storage system includes reading data elements from the data storage system. Clusters of data elements are formed with each data element being a member of at least one cluster. At least one data element is associated with two or more clusters. Membership of the data element belonging to respective ones of the two or more clusters is represented by a measure of ambiguity. Information is stored in the data storage system to represent the formed clusters.", + "c-en-0001": "A method implemented by data processing system for identifying one or more matches between a first data element, with a plurality of fields and with one or more values of one or more of the fields representing a key for the first data element, and each of one or more second data elements stored in a data storage system, the method including: accessing, from a hardware storage device, the first data element with the plurality of fields, with the one or more values of the one or more of the fields representing the key for the first data element, and with the first data element including one or more comparison fields used to corroborate one or more variant matches; accessing, from the hardware storage device, the one or more second data elements, with a second data element including one or more search fields specifying one or more search terms or one or more variants of one or more search terms, and one or more comparison fields used to corroborate the one or more variant matches; accessing one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, with a variant of the key being specified in accordance with a variant relation for the key; determining one or more variant matches between the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, and one or more respective values of one or more search fields of the one or more second data elements; and corroborating the one or more variant matches of the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element based on a comparison of one or more respective values of the one or more comparison fields of the one or more second data elements, to one or more values of the one or more comparison fields in the first data element for which the one or more variants of the key are accessed, with a comparison field being different from a search field.", + "c-en-0002": "The method of claim 1 , further including: joining the one or more second data elements with the first data element based on the corroborating.", + "c-en-0003": "The method of claim 1 , further including: returning the one or more second data elements based on the corroborating.", + "c-en-0004": "The method of claim 1 , further including: forming clusters of data elements with each data element being a member of at least one cluster, with at least one of the clusters including the one or more second data elements and the first data element.", + "c-en-0005": "The method of claim 4 , wherein at least one data element is associated with two or more clusters, with memberships of the at least one data element belonging to respective ones of the two or more clusters being represented by a measure of ambiguity.", + "c-en-0006": "The method of claim 5 , wherein each value of the measure of ambiguity representing a membership of the at least one data element belonging to a respective one of the two or more clusters is between zero and one.", + "c-en-0007": "The method of claim 5 , wherein values of the measure of ambiguity representing the memberships are related to a likelihood of the at least one data element belonging to the respective ones of the two or more clusters.", + "c-en-0008": "The method of claim 5 , wherein values of the measure of ambiguity representing the memberships are established based on a function, the function representing relationships between the at least one data element and the two or more clusters.", + "c-en-0009": "The method of claim 8 , wherein the relationships represented by the function are related to the likelihood of a data element belonging to respective ones of the two or more clusters.", + "c-en-0010": "The method of claim 4 , further including: performing a rollup operation that calculates a weighted subtotal of a quantity within a first cluster of the clusters, the quantity being associated with one or more data elements, and the weighted subtotal being calculated by summing within the first cluster products of respective values of the quantity associated with each of a plurality of data elements in the first cluster and a respective value of a measure of ambiguity representing the membership of the plurality of data elements in the first cluster.", + "c-en-0011": "The method of claim 10 , further including calculating an exclusive subtotal of the quantity and an inclusive subtotal of the quantity, the exclusive subtotal being calculated by excluding data elements in the first cluster that are associated with two or more clusters and the inclusive subtotal being calculated by including data elements in the first cluster that are associated with two or more clusters.", + "c-en-0012": "The method of claim 1 , further including determining membership of a given data element in a given cluster based on one or more values of one or more comparison fields of the given data element.", + "c-en-0013": "The method of claim 1 , further including: creating the key from the one or more values of the one or more of the fields in the first data element.", + "c-en-0014": "The method of claim 1 , wherein the hardware storage device incudes a volatile hardware storage device or a non-volatile hardware storage device.", + "c-en-0015": "A system for identifying one or more matches between a first data element, with a plurality of fields and with one or more values of one or more of the fields representing a key for the first data element, and each of one or more second data elements stored in a data storage system, the system including: one or more processors; and one or more machine-readable hardware storage devices storing instructions that are executable by the one or more processors to perform operations including: accessing, from a hardware storage device, the first data element with the plurality of fields, with the one or more values of the one or more of the fields representing the key for the first data element, and with the first data element including one or more comparison fields used to corroborate one or more variant matches; accessing, from the hardware storage device, the one or more second data elements, with a second data element including one or more search fields specifying one or more search terms or one or more variants of one or more search terms, and one or more comparison fields used to corroborate the one or more variant matches; accessing one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, with a variant of the key being specified in accordance with a variant relation for the key; determining one or more variant matches between the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, and one or more respective values of one or more search fields of the one or more second data elements; and corroborating the one or more variant matches of the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element based on a comparison of one or more respective values of the one or more comparison fields of the one or more second data elements, to one or more values of the one or more comparison fields in the first data element for which the one or more variants of the key are accessed, with a comparison field being different from a search field.", + "c-en-0016": "The system of claim 15 , wherein the operations further include: joining the one or more second data elements with the first data element based on the corroborating.", + "c-en-0017": "The system of claim 15 , wherein the operations further include: returning the one or more second data elements based on the corroborating.", + "c-en-0018": "The system of claim 15 , wherein the operations further include: forming clusters of data elements with each data element being a member of at least one cluster, with at least one of the clusters including the one or more second data elements and the first data element.", + "c-en-0019": "The system of claim 15 , wherein the hardware storage device incudes a volatile hardware storage device or a non-volatile hardware storage device.", + "c-en-0020": "One or more machine-readable hardware storage devices for identifying one or more matches between a first data element, with a plurality of fields and with one or more values of one or more of the fields representing a key for the first data element, and each of one or more second data elements stored in a data storage system, the one or more machine-readable hardware storage devices storing instructions that are executable by one or more processors to perform operations including: accessing, from a hardware storage device, the first data element with the plurality of fields, with the one or more values of the one or more of the fields representing the key for the first data element, and with the first data element including one or more comparison fields used to corroborate one or more variant matches; accessing, from the hardware storage device, the one or more second data elements, with a second data element including one or more search fields specifying one or more search terms or one or more variants of one or more search terms, and one or more comparison fields used to corroborate the one on more variant matches; accessing one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, with a variant of the key being specified in accordance with a variant relation for the key; determining one or more variant matches between the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element, and one or more respective values of one or more search fields of the one or more second data elements; and corroborating the one or more variant matches of the one or more variants of the key represented by the one or more values of the one or more of the fields of the first data element based on a comparison of one or more respective values of the one or more comparison fields of the one or more second data elements, to one or more values of the one or more comparison fields in the first data element for which the one or more variants of the key are accessed, with a comparison field being different from a search field.", + "c-en-0021": "The one or more machine-readable hardware storage devices of claim 20 , wherein the operations further include: joining the one or more second data elements with the first data element based on the corroborating.", + "c-en-0022": "The one or more machine-readable hardware storage devices of claim 20 , wherein the operations further include: returning the one or more second data elements based on the corroborating.", + "c-en-0023": "The one or more machine-readable hardware storage devices of claim 20 , wherein the operations further include: forming clusters of data elements with each data element being a member of at least one cluster, with at least one of the clusters including the one or more second data elements and the first data element.", + "c-en-0024": "The one or more machine-readable hardware storage devices of claim 20 , wherein the hardware storage device incudes a volatile hardware storage device or a non-volatile hardware storage device.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a continuation of and claims priority under 35 U.S.C. \u00a7120 to U.S. patent application Ser. No. 13/747,669, entitled \u201cFUZZY DATA OPERATIONS\u201d, filed Jan. 23, 2013, which is a continuation of and claims priority under 35 U.S.C. \u00a7120 to U.S. patent application Ser. No. 12/605,007 , entitled \u201cFUZZY DATA OPERATIONS\u201d, filed Oct. 23, 2009, which claims benefit from and priority to U.S. Provisional Patent Application No. 61/107,971, entitled \u201cFUZZY DATA OPERATIONS,\u201d filed on Oct. 23, 2008, now abandoned. The entire contents of each of the foregoing applications are hereby incorporated by reference.", + "p0003": "BACKGROUND", + "p0004": "This description relates to fuzzy data operations in the field of data management.", + "p0005": "Data operations, such as clustering, join, search, rollup, and sort, are employed in data management to handle data. Clustering is an operation that classifies data into different groups. Join combines two pieces of data together. Search by a key finds data entries that match that key. Rollup is an operation that calculates one or more levels of subtotals across a group of data. Sort is an operation that orders data.", + "p0006": "Data quality is important in data management. Mistakes or inaccuracies resulting from data operations degrade data quality. For example, classifying an employee of Corporation ABC, John Smith, as a temporary worker or a permanent worker entitles John Smith to a different level of benefits. Erroneous classification of John Smith's employment status, e.g., mistakes in data operation clustering, affects the quality of Corporation ABC's human resource data.", + "p0007": "Some implementations of data operations rely on exact comparison of field values to identify matching records, to define groups of related records or to link records. When data is ambiguous, imperfect, incomplete, or uncertain, methods based on exact comparison of field values may break down.", + "p0008": "When there is an inherent ambiguity associated with a data operation, for example, clustering, one approach to resolve the ambiguity may be simply to ignore the ambiguity and to force a piece of data into a particular group. For example, the employee of Corporation ABC, John Smith, works for both the marketing department and the R&D department. In Corporation ABC's human resource database, John Smith may be associated with either the marketing department or the R&D department, but often is associated with just one department. The forced classification of the piece of data into a particular group may mask the inherent ambiguity and adversely affect data quality.", + "p0009": "When there is an uncertainty associated with a data operation, for example, clustering, because of a pending outcome of an event, for example, a legal dispute between entity A and entity B that involves the ownership of a piece of an asset, forcing a piece of data into a particular group may not be the best approach to address the fluidity of the situation. Prior to the adjudication, the ownership of the asset is uncertain. Assigning the asset to either A or B may turn out to be inaccurate.", + "p0010": "When there is an uncertainty associated with a data operation, for example, rollup, because of an ambiguous identification of group membership, assigning membership to one group among several alternatives to preserve accounting integrity may give a misleading picture. For example, a bank may be interested in determining its exposure on loans to counterparties for risk assessment and regulatory purposes. Identification of a counterparty is often made by company name, which may lead to ambiguous identifications because of wide variability in the recorded form of a company's name. In turn, this means assignment of loan exposures to counterparties is ambiguous. It may happen that loans properly associated to one company become divided among several apparently distinct companies, which actually are simply variant forms of the name of the one company. This results in understating the exposure of the bank to any single counterparty. Alternatively, if an arbitrary selection among alternatives is made, an exposure may be falsely assigned to one counterparty when it properly belongs to another, perhaps overstating the exposure to the first and understating it to the second.", + "p0011": "When there is an uncertainty associated with a data operation, for example, join, because of incorrect or missing information, forcing a piece of data into a particular group or ignoring the piece of data may result in either a false association or loss of information. For example, when attempting to join tables from two different databases, there is often no common key shared by the database tables. To overcome this, data within the tables, e.g. customer address, is used to infer a relation between records in the two databases. Address information may however be incorrect or incomplete. Suppose address validation against a definitive reference set, like a Postal Address File, shows the house number on a record in table A is invalid while there are multiple addresses in table B which might be valid alternative completions of the address. Arbitrarily choosing a completion of the address in the record in table A may lead to a false association while ignoring the record leads to loss of information.", + "p0012": "When there is an ambiguity associated with a data operation, e.g. search, because of inaccurate data entry, one approach is to propose a single alternative or a simple list of alternative corrections. If this is part of the validation process of data being entered into a database by an operator, a single alternative when multiple alternatives exist may lead the operator into a false sense of security in accepting the correction. If a simple list of alternatives is provided, the operator may have no rational basis for choosing among the alternatives. If a single choice is required and some degradation of data quality is accepted for a wrong choice, then minimizing and quantifying the possible loss of data quality becomes the objective.", + "p0013": "SUMMARY", + "p0014": "In general, in one aspect, a method for clustering data elements stored in a data storage system includes reading data elements from the data storage system. Clusters of data elements are formed with each data element being a member of at least one cluster. At least one data element is associated with two or more clusters. Membership of the data element belonging to respective ones of the two or more clusters is represented by a measure of ambiguity. Information is stored in the data storage system to represent the formed clusters.", + "p0015": "Aspects may include one or more of the following features.", + "p0016": "Each value of the measure of ambiguity representing a membership of the data element belonging to a respective one of the two or more clusters may be between zero and one.", + "p0017": "Values of the measure of ambiguity representing the memberships may be related to the likelihood of the data element belonging to the respective ones of the two or more clusters.", + "p0018": "A sum of each value of the measure of ambiguity representing a membership of the data element belonging to a respective one of the two or more clusters may be one.", + "p0019": "The method may include preserving accounting integrity using values of the measure of ambiguity.", + "p0020": "Accounting integrity for a given quantity may be achieved by weighting the quantity with values of the measure of ambiguity.", + "p0021": "The method may include performing a data operation that uses values of the measure of ambiguity representing the memberships.", + "p0022": "The data operation may include a rollup that calculates a weighted subtotal of a quantity within a first cluster of the one or more clusters, the quantity being associated with the data element, and the subtotal being calculated by summing within the first cluster the products of the value of the quantity associated with each of the data elements in the first cluster and the respective value of the measure of ambiguity representing the membership of the data elements in the first cluster.", + "p0023": "The method may include calculating an exclusive subtotal of the quantity and an inclusive subtotal of the quantity, the exclusive subtotal being calculated by excluding the data elements in the first cluster that are associated with two or more clusters and the inclusive subtotal being calculated by including the data elements in the first cluster that are associated with two or more clusters.", + "p0024": "Values of the measure of ambiguity representing the memberships may be established based on a function, the function representing relationships between the data element and the two or more clusters.", + "p0025": "The relationships represented by the function may be related to the likelihood of the data element belonging to respective ones of the two or more clusters.", + "p0026": "The relationships represented by the function may be based on quantified similarities between the data element and elements representing respective ones of the two or more clusters.", + "p0027": "The elements representing the respective ones of the two or more clusters may be keys of the respective clusters.", + "p0028": "In some arrangements, values of the measure of ambiguity of the data element belonging to each cluster of the two or more clusters may be equal for each cluster.", + "p0029": "Values of the measure of ambiguity of the data element belonging to each cluster of the two or more clusters may be based on observed frequencies of the data element in a reference set.", + "p0030": "Each cluster of the two or more clusters may represent a different potential error in the data element. Values of the measured ambiguity of the data element belonging to each cluster of the two or more clusters may be based on the likelihood of the potential error in the data element represented by each cluster.", + "p0031": "Forming data clusters may include forming a plurality of superclusters of data elements and for each supercluster forming clusters of data elements within the supercluster.", + "p0032": "Forming each supercluster may include determining matches between objects in different data elements based on a variant relation between the objects in the different data elements.", + "p0033": "The variant relation between a first object and a second object may corresponds to a value of a function representing a distance between the first object and the second object being below a predetermined threshold.", + "p0034": "In some arrangements, variant relation may not be an equivalence relation.", + "p0035": "At least one data element may be in more than one supercluster.", + "p0036": "In another aspect, in general, a system for clustering data elements stored in a data storage system includes: means for reading data elements from the data storage system; means for forming clusters of data elements with each data element being a member of at least one cluster; means for associating at least one data element with two or more clusters, with memberships of the data element belonging to respective ones of the two or more clusters being represented by a measure of ambiguity; and means for storing information in the data storage system to represent the formed clusters.", + "p0037": "In another aspect, in general, a computer-readable medium storing a computer program for clustering data elements stored in a data storage system is described. The computer program includes instructions for causing a computer to: read data elements from the data storage system; form clusters of data elements with each data element being a member of at least one cluster; associate at least one data element with two or more clusters, with memberships of the data element belonging to respective ones of the two or more clusters being represented by a measure of ambiguity; and store information in the data storage system to represent the formed clusters.", + "p0038": "In another aspect, in general, a method for performing a data operation that receives a key and returns one or more data elements from a data storage system includes determining multiple candidate data elements based on candidate matches between the key and values of one or more search fields of the data elements. The candidate matches are corroborated based on values of one or more comparison fields of the candidate data elements different from the search fields.", + "p0039": "Aspects may include one or more of the following features.", + "p0040": "The data operation may include forming clusters of data elements with each data element being a member of at least one cluster.", + "p0041": "At least one data element may be associated with two or more clusters, with memberships of the data element belonging to respective ones of the two or more clusters being represented by a measure of ambiguity.", + "p0042": "The data operation may include a rollup that calculates a weighted subtotal of a quantity within a first cluster of the one or more clusters, the quantity being associated with the data element, and the subtotal being calculated by summing within the first cluster the products of the value of the quantity associated with each of the data elements in the first cluster and the respective value of the measure of ambiguity representing the membership of the data elements in the first cluster.", + "p0043": "The method may also include calculating an exclusive subtotal of the quantity and an inclusive subtotal of the quantity, the exclusive subtotal being calculated by excluding the data elements in the first cluster that are associated with two or more clusters and the inclusive subtotal being calculated by including the data elements in the first cluster that are associated with two or more clusters.", + "p0044": "Each value of the measure of ambiguity representing a membership of the data element belonging to a respective one of the two or more clusters may be between zero and one.", + "p0045": "Values of the measure of ambiguity representing the memberships may be related to the likelihood of the data element belonging to the respective ones of the two or more clusters.", + "p0046": "Values of the measure of ambiguity representing the memberships may be established based on a function, the function representing relationships between the data element and the two or more clusters.", + "p0047": "The relationships represented by the function may be related to the likelihood of the data element belonging to respective ones of the two or more clusters.", + "p0048": "The method may further include determining membership of a given data element in a given cluster based on the values of the one or more comparison fields of the given data element.", + "p0049": "In another aspect, in general, a system for performing a data operation that receives a key and returns one or more data elements from a data storage system includes: means for determining multiple candidate data elements based on candidate matches between the key and values of one or more search fields of the data elements; and means for corroborating the candidate matches based on values of one or more comparison fields of the candidate data elements different from the search fields.", + "p0050": "In another aspect, in general, a computer-readable medium storing a computer program for performing a data operation that receives a key and returns one or more data elements from a data storage system is described. The computer program includes instructions for causing a computer to: determine multiple candidate data elements based on candidate matches between the key and values of one or more search fields of the data elements; and corroborate the candidate matches based on values of one or more comparison fields of the candidate data elements different from the search fields.", + "p0051": "In another aspect, in general, a method for measuring data quality of data elements in a data storage system includes reading data elements from the data storage system. A value of a measure of ambiguity for the entry; in computed for each of one or more entries in one or more fields of the data elements. A representation of data quality of the data elements in the data storage system based on the values of the measure of ambiguity is outputted.", + "p0052": "Aspects may include one or more of the following features.", + "p0053": "Computing the value of the measure of ambiguity may include comparing the entries in one or more fields of the data elements to reference values. One or more variants for at least a first entry that is not an exact match to a reference value may be identified. The value of the measure of ambiguity for the first entry may be computed based on the variants for the first entry.", + "p0054": "The value of the measure of ambiguity for the first entry may be based on the number of variants for the first entry.", + "p0055": "The representation of data quality of the data elements in the data storage system may include a histogram plot of the number of entries having a specified number of variants.", + "p0056": "The specified number of variants may be specified as being within a range.", + "p0057": "The representation of data quality of the data elements in the data storage system may include a list of entries having number of variants larger than a predetermined threshold.", + "p0058": "Computing the value of the measure of ambiguity may include determining respective frequencies of different entries in the one or more fields. The value of the measure of ambiguity for a first entry may be computed based on a relative frequency of the first entry compared to frequencies of other entries.", + "p0059": "In another aspect, in general, a system for measuring data quality of data elements in a data storage system includes: means for reading data elements from the data storage system; means for computing, for each of one or more entries in one or more fields of the data elements, a value of a measure of ambiguity for the entry; and means for outputting a representation of data quality of the data elements in the data storage system based on the values of the measure of ambiguity.", + "p0060": "In another aspect, in general, a computer-readable medium storing a computer program for measuring data quality of data elements in a data storage system is described. The computer program including instructions for causing a computer to: read data elements from the data storage system; for each of one or more entries in one or more fields of the data elements, compute a value of a measure of ambiguity for the entry; and output a representation of data quality of the data elements in the data storage system based on the values of the measure of ambiguity.", + "p0061": "In another aspect, in general, a method for joining data elements from two or more datasets stored in at least one data storage system include determining matches between objects in data elements from a first dataset and objects in data elements from a second dataset based on a variant relation between the objects in the data elements from the first dataset and objects in the data elements from the second dataset. Respective data elements having respective objects determined as matches are evaluated. The data to elements from the first dataset are joined with the data elements from the second dataset based on the evaluation of data elements.", + "p0062": "Aspects can include one or more of the following features.", + "p0063": "The variant relation between a first object and a second object may corresponds to a value of a function representing a distance between the first object and the second object being below a predetermined threshold.", + "p0064": "The variant relation may not be an equivalence relation.", + "p0065": "Determining a match between an object in a first data element from the first dataset and an object in a second data element in the second dataset may include determining that the variant relation holds between the object in the first data element and the object in the second data element.", + "p0066": "Determining a match between an object in a first data element from the first dataset and an object in a second data element in the second dataset may include determining: that the variant relation holds between the object in the first data element and an object in a third data element in the first dataset, and that the variant relation hold between the object in the third data element and the object in the second data element.", + "p0067": "Evaluating respective data elements having respective objects determined as matches may include comparison of objects in the respective data elements other than the respective objects determined as matches.", + "p0068": "In another aspect, in general, a system for joining data elements from two or more datasets stored in at least one data storage system includes: means for determining matches between objects in data elements from a first dataset and objects in data elements from a second dataset based on a variant relation between the objects in the data elements from the first dataset and objects in the data elements from the second dataset; means for evaluating respective data elements having respective objects determined as matches; and means for joining the data elements from the first dataset with the data elements from the second dataset based on the evaluation of data elements.", + "p0069": "In another aspect, in general a computer-readable medium storing a computer program for joining data elements from two or more datasets stored in at least one data storage system is described. The computer program including instructions for causing a computer to: determine matches between objects in data elements from a first dataset and objects in data elements from a second dataset based on a variant relation between the objects in the data elements from the first dataset and objects in the data elements from the second dataset; evaluate respective data elements having respective objects determined as matches; and join the data elements from the first dataset with the data elements from the second dataset based on the evaluation of data elements.", + "p0070": "DESCRIPTION OF DRAWINGS", + "p0071": "FIG. 1 is a block diagram of a system for executing graph-based computations.", + "p0072": "FIG. 2 A is an example of a data element belonging to multiple clusters.", + "p0073": "FIG. 2 B is an example of an operation performed on a cluster.", + "p0074": "FIG. 2 C, 2 D are examples of distance calculation.", + "p0075": "FIG. 3 is an illustration of fuzzy clusters.", + "p0076": "FIG. 4 is another illustration of fuzzy clusters", + "p0077": "FIG. 5 is a flow chart of how to generate fuzzy clusters.", + "p0078": "FIG. 6 illustrates an example of fuzzy search.", + "p0079": "DESCRIPTION", + "p0080": "The techniques for performing fuzzy data operations can be applied to a variety of types of systems including different forms of database systems storing datasets. As used herein, a dataset includes any collection of data that enables portions of data to be organized as records having values for respective fields . The database system and stored datasets can take any of a variety of forms, such a sophisticated database management system or a file system storing simple flat files. One aspect of various database systems is the type of record structure it uses for records within a dataset . In some systems, the record structure of a dataset may simply define individual text documents as records and the contents of the document represent values of one or more fields. In some systems, there is no requirement that all the records within a single dataset have the same structure .", + "p0081": "Complex computations can often be expressed as a data flow through a directed graph , with components of the computation being associated with the vertices of the graph and data flows between the components corresponding to links of the graph. A system that implements such graph-based computations is described in U.S. Pat. No. 5,966,072, EXECUTING COMPUTATIONS EXPRESSED AS GRAPHS, incorporated herein by reference. One approach to executing a graph-based computation is to execute a number of processes, each associated with a different vertex of the graph, and to establish communication paths between the processes according to the links of the graph. For example, the communication paths can use TCP/IP or UNIX domain sockets, or use shared memory to pass data between the processes.", + "p0082": "Referring to FIG. 1 , a system 10 for executing graph-based computations includes a development environment 14 coupled to a data store 12 and a runtime environment 18 coupled to the data store 12 . A developer 11 builds applications using the development environment 14 . An application can be associated with one or more dataflow graphs specified by data structures in the data store 12 which may be written to the data store as a result of the developer's use of the development environment 14 . A data structure 13 for a computation graph 15 specifies, for example, the vertices of a computation graph and links between the vertices. The data structures can also include various characteristics of the components, datasets, and flows of the dataflow graphs.", + "p0083": "The runtime environment 18 may be hosted on one or more general-purpose computers under the control of a suitable operating system, such as the UNIX operating system. For example, the runtime environment 18 can include a multiple-node parallel computing environment including a configuration of computer systems using multiple central processing units , either local , or locally distributed , or remotely, or remotely distributed , or any combination thereof.", + "p0084": "The runtime environment 18 is configured to receive control inputs from the data store 12 and/or from a user 17 for executing and configuring computations. The control inputs can include commands to process particular datasets using corresponding dataflow graphs, which are specified in the stored graph data structures. The user 17 can interact with the runtime environment 18 , for example, using a command line or graphical interface.", + "p0085": "The runtime environment 18 includes a pre-execution module 20 and an execution module 22 . The pre-execution module 20 performs any pre-processing procedures and prepares and maintains resources for executing computation graphs, such as a dictionary 21 and an archive 24 used for various fuzzy operations . The dictionary 21 stores words and associated information about words appearing in a dataset. The archive 24 stores various results from pre-processing based on words, phrases, or records of the dataset. The dictionary 21 and archive 24 can be implemented in any of a variety of formats and can be organized as single collections of data or as multiple dictionaries and archives. The execution module 22 schedules and controls execution of the processes assigned to a computation graph for performing the computations of the components. The execution module 22 can interact with external computing resources coupled to the system 10 that are accessed during processing associated with the graph components, such as a data source 26 providing records from a database system. The fuzzy operations performed in the system 10 can be used for various purposes such as analyzing data to assess its quality or to organize and/or consolidate the data.", + "p0086": "A core asset of any business or other organization is the data it holds in order to conduct its operations, from lists of products, services and customers to transactions, contracts and accounts with individuals, banks and other businesses. This data is stored in multiple formats and in multiple systems, from paper and spreadsheets to relational databases and enterprise applications, like accounting or supply-chain management systems. A central concern of every organization is the quality and integrity of this data.", + "p0087": "If invoices contain incorrect prices or mislabeled products, the wrong amount is charged or the wrong item is delivered. If addresses for customers or suppliers are wrong, shipments or orders may be delayed or lost and invoices or payments may not reach their intended party. If the key representing a customer on one system links to the accounts of a different customer on another system, reports on the state of the customer's accounts will be unreliable, and, even worse, a customer may have access to the accounts of other customers. Poor data quality disrupts the orderly conduct of business and may result in lost income, wounded reputation or missed opportunities.", + "p0088": "An important subset of the data of a business or organization is its non-transactional reference data, sometimes referred to as its master data. This can include lists of products, customers, accounts, suppliers, and the specific valid values used to represent particular attributes of each data item . Generally, master data excludes the short-term operational data of the organization, like transactions and prices. Master data management is concerned with the organization and maintenance of the reference data of an organization. One of its main concerns is with the quality and integrity of the reference data.", + "p0089": "Problems with data quality and referential integrity take many forms. These problems are exacerbated by the existence of multiple data systems of different kinds, which may be difficult to keep consistent. A non-exhaustive list of potential problems follows.", + "p0090": "1) Data may be entered or recorded incorrectly: the entry made is not the one intended. There may be typographical or transcription errors in the entry, resulting in variant spellings of words, for example, in customer names or addresses, product labels or descriptions, or values expected to be taken from an enumerated list. Many data entry applications have safeguards intended to validate data on entry to prevent such errors, but they still occur.", + "p0091": "2) Data may be incomplete: not all fields are populated. A customer application may have had information missing from certain fields. The completion of a form may have been interrupted during entry. Information may have been deemed invalid on entry and discarded. Information to complete the entry may not have been available at the time of entry, perhaps pending completion of some other activity.", + "p0092": "3) Data may be invalid: a field is populated but with an invalid value. An entry may not match any of the values expected from an enumerated list. An entry might not be valid for its expected data type, for example, there may be alphabetic characters in a decimal field or the day of a month in a date may be larger than the number of days in that month .", + "p0093": "4) Data may be entered in the wrong field . A city or zip code may appear in the street field of an address. A foreign address with a different format than expected may have been forced to fit the expected format. The product id may be in a description or comment field on an invoice or order form. The first and last names of an individual may be swapped if the surname is a common first name or if the names are unfamiliar or foreign.", + "p0094": "5) Standards for data entry may not exist: data may be entered inconsistently. The order of the lines of an address is not standardized and may not always be recorded in the same way, even in the same dataset. The detailed form of a company's name is not standardized and a number of variant forms may be acceptable, even in the same dataset. A customer name may contain a full middle name, or the middle name may be absent, or there may only be a middle initial. Similarly, the first name may only be an initial. \u201cDouble-barreled\u201d last names may be present in the last name with or without a hyphen or may be split between the middle and last name fields.", + "p0095": "6) Data may be held in free text fields. Remarks in comment fields on invoices or order forms may contain important information, like a product name or descriptive attributes, which would otherwise be missing. Description fields on a database table may contain an explanation for changes to other fields, for example, when a woman's last name changes because of marriage.", + "p0096": "7) Key relations may be broken. Databases use keys to link related data held in different tables and sometimes in tables in different databases. When keys do not properly link the correct records, the referential integrity of the database has been broken. Keys may incorrectly link records, as when one customer is linked to the accounts properly belonging to another customer. Keys may link to non-existent records, for example, the customer key on an account record does not link to any existing customer record. In some cases, the customer record exists but has a different key; the key is sometimes described as a \u201cmissing key.\u201d In other cases, when no corresponding customer record exists at all, the account record is said to be orphaned.", + "p0097": "8) Key relations may not exist. Databases with different origins can hold similar data, but no keys exist to link data shared by them. It is not uncommon for one line of business not to realize it shares customers with another line of business. When businesses or organizations merge, the master data of the two entities may be combined. The different standards and nonequivalent valid values of the two entities make it difficult to achieve a consistent set of master data, but the problem of identifying and linking shared data, like customers, is often harder.", + "p0098": "Data cleansing seeks to identify and correct many of these issues. Because of the number and complexity of legacy systems, the number of interfaces between systems, and the rate of introduction of new systems, the real challenge is often not how to fix the problems of data quality, but how to cope with them.", + "p0099": "Perhaps the central concept in finding, accessing and manipulating data in the systems of a business or organization is that of a \u201ckey.\u201d A primary key is a field, or combination of fields, whose value serves to identify a record uniquely in a dataset. Within a relational database, every table may have a primary key which identifies records uniquely within the table . Foreign keys in a table are keys that link to records in other tables.", + "p0100": "A number of data operations can be performed which depend upon the keys of a database table or other dataset. The familiar key-based data operations are lookup, join, rollup, scan, sort, merge and, in parallel processing, partition by key. These data operations are based on exact agreement of keys, called here \u201cexact matching.\u201d In the data operation lookup, a key is used to retrieve one or more records from a lookup dataset having an exactly matching key. In the data operation join, two datasets are combined a record at a time by adjoining the contents of a record from one dataset with the contents of a record sharing a common key from another dataset . If more than one record has a matching common key, separate output records are formed for each matching record pair.", + "p0101": "In the data operation rollup, the contents of a group of records sharing a common key are combined to produce a single output record having the same key. For example, a customer's total transaction amount would be obtained by rolling up transaction records to the customer level while summing the transaction amounts. In the data operation scan, for each record in a group of records sharing a common key, an output record is computed using the contents of all previously seen having the same key. For example, a running total of customer spending may be computed with a scan of transactions by a customer.", + "p0102": "In the data operation sort, records are ordered by their key values. In the data operation merge, sorted data from one or more data streams are combined into a single stream such that the output stream is also sorted. In the parallel processing data operation partition by key, data is allocated to processing partitions based on the value of the key.", + "p0103": "When multiple independent systems coexist, each of which may have data quality problems of the kinds discussed above, keys relating records with common data do not generally exist, and keys that do exist may not be reliable. Ultimately the data within each record are the items of interest. The key can be thought of as a convenient fiction introduced in a database to identify and access the data. In the absence of a reliable key, the data itself may be used for the purpose of identification.", + "p0104": "Accessing records through their contents may be based on searching. For example, customers in one database may be sought in a second database by name. Since name is an ambiguous identifier, it is rarely a key. While a name can be used to initiate identification, supporting information such as birth date and address are generally needed to corroborate the match.", + "p0105": "Furthermore, because of data quality issues, often neither the name nor the corroborating information need agree exactly for a record to be a correct match. Exact agreement may be too restrictive, and demanding a precise match may result in many correct identifications being missed. The data operation of search retrieves data entries that match closely but not necessarily exactly. For example, a fuzzy search for \u201cLeslie\u201d may return a record for a person named \u201cLesley.\u201d In a fuzzy search, there may be more than one matching record having differing degrees of similarity or corroboration . Candidate matches may not be corroborated sufficiently to qualify as definitive or even acceptable matches. For example, the birth date of a retrieved Lesley record may not agree with the birthdate of the Leslie record, in which case the candidate match is not corroborated.", + "p0106": "When searching the single step process of using an exact key for a lookup is replaced by a two step process: Records are identified for retrieval using search terms and assessed to determine a match. Search terms are not keys since they rarely uniquely identify a record; however, they are used like keys to link records.", + "p0107": "For clarity, it is useful to distinguish the field from which search terms are chosen from the fields used to compare records to assess the quality of the match. These may be called search fields and comparison fields, respectively.", + "p0108": "To find and judge matches when search terms or comparison fields are not identical, scoring functions may be used to recognize variant values. Candidate matches may be retrieved using variant search terms and evaluated using scoring functions to quantify the quality of the match between corroborating fields. These scoring functions are designed to account for various data quality issues. They recognize a match despite these issues, although with reduced score. For example, a scoring function for personal names may tolerate exchanging first and last names or using a middle initial while a scoring function tuned for company names might place more importance on word order than on missing words.", + "p0109": "Another fundamental use of exact keys is to identify sets of records having a common key value, often called keygroups. These keygroups play a central role in many key-based data operations. When the requirement of exactly matching keys is relaxed, the question arises of how to group keys. A set of keys grouped together based on a relaxed matching criteria is called a cluster.", + "p0110": "Generally, a cluster may be a set of records whose comparison fields meet a comparison test: For example, in one arrangement, a record is a member of cluster if its score with the cluster exceeds a threshold. There are many different ways to define the score of a record with a cluster, typically but not exclusively involving individually scoring the record with each member of the cluster and then combining the scores. For example, the score might be the maximum of the score of the record with each member of the cluster, or it might be the average of the scores with each member of the cluster. In some arrangements, scoring a pair of records involves assigning a number to the result of comparing one set of field values with the other. The comparison of field values may contain both qualitative and quantitative assessments.", + "p0111": "The issue in defining clusters is that ambiguous membership assignments are possible because comparison of field values is a scored relation. In particular, scoring may indicate one piece of data belongs to more than one cluster. In one arrangement, this ambiguity may be handled by forcing the piece of data into one of the clusters in order to make the clusters sharply defined, as they are in the exact key case. In which case, the key-based data operations remain essentially as they are in the exact key case.", + "p0112": "Exact-key-based data operations may not always be as precise or accurate as desired for various reasons. One reason may be the inherent ambiguity associated with a piece of data or with a data operation. For example, a piece of data may legitimately belong to more than one group. Under some clustering methods, inherent ambiguity can make accurate classification difficult or unattainable. For example, in the above mentioned human resource database, in which an employee is to be classified according to the department to which he or she belongs, the employee may belong to two departments at the same time, such as marketing and R&D. Forcefully associating the employee with either department, marketing or R&D, may be misleading. Simply associating the employee with both departments may cause double-counting problems. For instance, expenses, such as medical, may be tallied twice for the same employee.", + "p0113": "Another reason that precise classification may not be possible is that the outcome of a pending event may impact the current classification. For example, an organization's legal status as a charitable organization or non-charitable organization may alter its tax obligations. Further suppose there is ongoing litigation between the IRS and the organization regarding whether that organization qualifies as a charitable organization and therefore deserves a tax deduction. If, in the organization's annual budget, the tax status of the organization is presumed to be that of a charitable organization and accordingly a smaller budget is set aside for tax payments, and if later the court decides that the organization is a non-charitable organization, and therefore cannot take the tax deduction entitled only to a charitable organization, the annual budget has to be revamped. The traditional way of handling such a situation is often by appending a note to the budget explaining the possible adverse court decision that may affect the budget. In the event that the adverse court decision occurs the budget has to be revised. But worse than having to correct the budget, if the budget has been used by other applications, in other business areas, or in other countries, corrections beyond the budget itself may be impossible because the ripple effects may be impossible to trace.", + "p0114": "The above two examples illustrate how conventional data approaches may not be adequate for handling ambiguous assignments to clusters . Ambiguous assignments arise when a one-to-one match to a cluster cannot be ascertained or may be better not to be ascertained. One challenge presented by allowing assignments to multiple clusters is how to preserve accounting integrity. The method of partial membership can be used for this purpose and will be discussed in detail later in this disclosure. To handle clusters containing elements with ambiguous membership, fuzzy data operations may be used.", + "p0115": "When clusters have elements with overlapping membership and some data is associated to more than one cluster, the data operation of fuzzy rollup may be used to perform calculations while preserving accounting integrity and reporting the range of error associated with possible alternative assignments. In the human resources example, a fuzzy rollup operation may be used to total expenses by department. If an employee works for more than one department expenses for that employee may be allocated among the departments, reflecting the employee's partial membership.", + "p0116": "When cluster membership is either conditional on future events, as in the legal example above, or uncertain because of ambiguous or incomplete information, as in the determination of counterparties in the banking example above, then a fuzzy rollup operation to compute, for example, monetary totals by group, should reflect this uncertainty while preserving accounting integrity. Certainly, in the case of an uncertain future event, such as the tax classification of a company, one eventuality does occur. Premature assignment to a particular alternative may give a misleading picture for the purposes of planning and risk assessment.", + "p0117": "For example, in FIG. 2 A , it is uncertain whether data element 120 belongs to cluster 122 , cluster 124 , or cluster 126 . It may be that the data element 120 belongs to the three clusters 122 , 124 , and 126 at the same time. It may also be that the data element 120 belongs to one cluster at one particular time but rotates among the three clusters. The memberships of data element 120 belonging to clusters 122 , 124 , and 126 are represented by n 1 , n 2 , and n 3 . n 1 , n 2 , and n 3 are fractional numbers. In the case where the data element 120 belongs to the three clusters at the same time with equal probabilities, n 1 , n 2 , and n 3 each may be assigned a fractional number \u2153. In this case, the sum of the partial memberships of data element 120 belonging to cluster 122 , 124 , and 126 is one . In an example in which the data element 120 belongs to one cluster at one particular time but rotates among the three clusters, at time t 1 , n 1 , n 2 , and n 3 may be of values 1, 0, and 0. At time t 2 , n 1 , n 2 , and n 3 may be of values 0, 1, and 0. The values of n 1 , n 2 , and n 3 may vary, but the sum of their values should always be one.", + "p0118": "In the banking example, knowing the maximum and minimum exposures to each counterparty, based on alternative assignments of exposures to ambiguously identified counterparties, gives a more complete picture of the possible exposure to any given counterparty and communicates the uncertain state of knowledge. Current beliefs about the future or on the likely resolution of ambiguities can be incorporated by weighting the assignment of members to clusters using probable likelihoods of membership, and these weights may be refined over time to reflect changing state of knowledge.", + "p0119": "The operation of fuzzy join enables two or more datasets to be combined when they do not share a common exact key. For instance, customer household records from different databases can be joined on address when the addresses are not literally identical. When an address from one dataset is incomplete or inaccurate, there may be multiple records in the second dataset for which it is a candidate match. The fuzzy join accommodates this possibility.", + "p0120": "A sort operation orders records by key and is often used prior to a key-based operation, like rollup or join, which acts on groups of records. A fuzzy sort may be used to order records prior to an operation such as a fuzzy rollup when individual records may be members of multiple clusters. The notion of sort order and the action of sorting is extended by replicating individual records which are ambiguous members of multiple clusters and positioning them in each of their associated clusters in the final ordering.", + "p0121": "Fuzzy data operations differ from conventional data operations in that in lieu of keygroups based on exactly matching keys, clusters are used. Clusters include the above example of retrieving Lesley when the key is Leslie. Clusters also include the example of classifying John Smith into the marketing department even though he does not exactly belong to the department because he only works in that department half of the time.", + "p0122": "FIG. 2 B illustrates am exemplary fuzzy data operation. In this example, fuzzy data operation 180 operates on a key 160 and retrieves data set 150 . Key 160 is a conventional key. The retrieved data set 150 includes 5 data elements, data element 151 , data element 152 , data element 153 , data element 154 , and data element 155 . These five data elements do not match key 160 . But nevertheless, they are retrieved by the data operation. This is where a fuzzy data operation differs from a conventional data operation. Given a key, a conventional data operation retrieves those data that match the key exactly. But a fuzzy data operation can retrieve data that do not match the key exactly.", + "p0123": "Fundamental to the definition of the clusters underlying fuzzy data operation is the comparison of data in different records. A comparison test is used to determine which records belong to each cluster. In some arrangements, the comparison test is a scored function of selected field values taken from each record, and the quantified difference between two pieces of data may be a distance.", + "p0124": "Distance Between Two Pieces of Data", + "p0125": "Differences between two pieces of data are often intuitively simple. For example, the difference between the names Leslie and Lesley is apparent and the difference between a full time employee and a part time employee is evident. However it is not always straightforward to quantify or measure differences between two pieces of data. Here we will briefly discuss two methods that can be used to measure a distance between two pieces of data. It should be understood that other methods quantifying differences between data can be readily developed based on the principles described below. Additional examples of fuzzy matching techniques and distance measures are described, for example, in U.S. Application Publication No. 2009/0182728, incorporated herein by reference.", + "p0126": "Distances Between Two Words.", + "p0127": "A method of measuring the distance between two words , often referred to as the \u201cedit distance,\u201d involves counting how many character operations it takes to get from one word to the other. In this example, a character operation involves a single character. A character can be encoded in any of a variety of ways. For example, it can be encoded using any single-byte or multi-byte encoding or code-point used to represent the character in a character set. The Levenshtein edit distance counts the number of character insertions, deletions and substitutions required to turn one word into another.", + "p0128": "A limitation of the Levenshtein edit distance and its variants is that they cannot be used in the online fuzzy match context, that is, when you have a query word that has not been seen before and want to find matching variants in an existing reference set. A deletion algorithm for computing variants can be applied instead. In this method, the distance between words is determined by counting the number of deletions required from each word to reach a matching word. FIG. 2 C shows how the deletion distance between Leslie and Lesley is computed. Operation 102 deletes \u201ci\u201d from \u201cLeslie\u201d to obtain \u201cLesle\u201d. Operation 104 deletes \u201cy\u201d from \u201cLesley\u201d to obtain \u201cLesle\u201d. The distance between \u201cLeslie\u201d and \u201cLesley\u201d is 1+1 .", + "p0129": "In some arrangements, more refined scoring can be made by comparing the positions and relative values of deleted characters. This allows for weighted scoring, where different weights are applied for different kinds of changes. For example, a substitution may be less important than a transposition or substitution of an \u201cm\u201d by an \u201cn\u201d is less important than an \u201cm\u201d by a \u201ck\u201d.", + "p0130": "The deletion algorithm can be used for online fuzzy search of a reference dataset in the following way. A deletion dictionary is constructed from the reference dataset by forming every word obtained by deleting one or more characters, to as many deletions considered necessary, from each word in the reference dataset. Both the original word and the positions of deleted characters are recorded along with the word resulting from the deletions. When a search is made, the query word is processed to construct every word obtained by deleting one or more characters. Each of these words is looked up in the reference deletion dictionary to find the corresponding original word. The matched original words may then be used in an ordinary exact search/lookup in the dataset. To reiterate, this method works even when the query word is a variant that does not occur in the reference dataset.", + "p0131": "Another example is the distance between \u201cCorp.\u201d and \u201cCo.\u201d. From \u201cCorp.\u201d to \u201cCo.\u201d, two deletions in one word are needed, a deletion of letter r and a deletion of letter p. Therefore if the distance between two words is defined as how many deletion operations are needed on each word to obtain a matching word, the distance between \u201cCorp.\u201d and \u201cCo.\u201d may be 2+0, even though \u201cCorp.\u201d and \u201cCo.\u201d are two interchangeable abbreviations for the same word \u201ccorporation.\u201d A conventional method that relies on exactly matching words will not yield satisfactory results in a case in which the data entered by a user uses \u201cCorp.\u201d while the key used by a data operation uses \u201cCo.\u201d. For example, a conventional search data operation that retrieves only data entries that match a key exactly will not yield ABC Corp. if the key used is ABC Co. Under fuzzy data operations, a fuzzy search can be constructed to return data entries that are within a certain distance of the key, e.g., 2+0 or better. Under such a fuzzy search, ABC Corp. may be returned as a match for the key ABC Co.", + "p0132": "Alternatively, since these two words are interchangeable as synonyms, the distance between \u201cCorp.\u201d and \u201cCo.\u201d may be defined as zero. A fuzzy search can be constructed to return data entries that contain user-specified synonyms. This example showcases the complexities with which fuzzy operations may need to deal.", + "p0133": "In the above examples, distances are computed based on operations such as insertion or deletion of a character, with both insertion and deletion counted as one operation. In other arrangements, distances can be computed based on weighted operations. The weighting can be used to bias one type of operation, for example, insertion, against another type of operation, for example, deletion. Alternatively, the weighting can be used to bias one individual operation against another individual operation. For example, an operation corresponding to deletion of a space may be weighted less than an operation corresponding to insertion of the letter z, to reflect the fact that an omission of a space is a common misspelling error while an insertion of the letter z in an English word is probably less a misspelling error than a true difference between two English words.", + "p0134": "For example, the distance between \u201csunshine\u201d and \u201csun shine\u201d is one insertion of a space. The distance between \u201czinc\u201d and \u201cInc\u201d is one insertion of the letter z. These two distances are equal, one operation, if individual operations are not weighted in calculating the distance. When a fuzzy search operation is constructed to return any matches that are within a distance of one operation, a search by key \u201csunshine\u201d will return \u201csun shine\u201d and a search by key \u201cInc\u201d will return \u201czinc\u201d.", + "p0135": "But if weighted operations are used, the two distances, the distance between \u201csun shine\u201d and \u201csunshine\u201d and that between \u201czinc\u201d and \u201cInc\u201d, can be different. For example, an insertion of a space may be weighted by a factor of 0.5 to reflect the fact that an insertion of a space is more likely caused by a typo. An insertion of the letter z may be weighted by a factor of 1 to reflect the fact that an extra letter z is less likely added by mistake. FIG. 2 D shows that when the operations are weighted as such, the distance between \u201csun shine\u201d and \u201csunshine\u201d is 0.5 operation while the distance between \u201cZinc\u201d and \u201cInc\u201d is one operation.", + "p0136": "In an arrangement in which a fuzzy search data operation is constructed to return any matches that are within the distance of 0.5 character operations of the key, a search of key \u201csunshine\u201d will return \u201csun shine\u201d. But a search of key \u201cInc\u201d will not return \u201cZinc\u201d.", + "p0137": "In some arrangements, more elaborate weighting options may be defined.", + "p0138": "Distances Between Two British Postal Codes", + "p0139": "Another application in which fuzzy matching is useful is working with a company's customer address database that contains duplicate records for the same household. The multiple entries for the same household may be caused by a typographical error in the zip code associated with the household, or may be caused by a misspelling of the name associated with the household.", + "p0140": "Possible typographical errors may include omitting or inserting a space, omitting or inserting a letter, and mis-typing a letter. It is less likely that a user makes two typographical errors in the same zip code, though not uncommon. It is unlikely that a user makes three typographical errors in the same zip code, though not impossible.", + "p0141": "FIG. 3 shows the possible duplicated records caused by typographical errors in a zip code. In customer address database 300 , there are six entries under the name John Smith: John Smith ox26qt; John Smith ox2 6qt; John Smith ox26qy; John Smith ox2 6qy; John Smith ox2 6qx; and John Smith ox2 7qy. The distance between every pair of records is labeled next to the line connecting the pair of records.", + "p0142": "Suppose that the company has decided that any record that contains a zip code within a deletion distance of 1+1 away from the zip code in a genuine record is most likely a spurious record, a record entered by mistake, and will be treated as a duplicate of the genuine record. Further suppose that the company has defined a fuzzy search to search for all the records that are within a deletion distance of 1+1 of the search key.", + "p0143": "A word is a variant of another word if the former is within a specified distance of the latter. The latter is referred to as the primary. In the present example, the specified deletion distance is 1+1 . The distance information between each postcode in the customer address database 300 is listed in FIG. 3 . Based on FIG. 4 , we can determine the variants for each record, as shown in FIG. 4 .", + "p0144": "FIG. 4 is a visualization tool and it is formed by representing each record with a box of a distinct shade and overlapping each record's box with the boxes of its variants. For example, record A's box overlaps with records B, C, and E's boxes because records B, C, and E are variants of record A. Record E's boxes overlaps with records A and F's boxes because records A and F are variants of record E.", + "p0145": "In some instances, the company may know which record is a genuine record while in some other instances; the company may not know which one is genuine.", + "p0146": "In a first example, the company knows that the genuine record is \u201cJohn Smith ox2 6qy\u201d. Running the fuzzy search using \u201cox2 6qy\u201d as the search key will retrieve the following two records: \u201cJohn Smith ox2 6qt\u201d and \u201cJohn Smith ox26qy\u201d. The company will treat these two records in the same cluster as duplicates of the genuine record \u201cJohn Smith ox2 6qy\u201d. The company may decide to eliminate these two duplicates or to group the three records together by assigning them a common key. This group is an example of a fuzzy cluster.", + "p0147": "Fuzzy clustering is a data operation that groups together the data having keys that do not exactly match but are within a certain distance from each other. Fuzzy clustering may be related to fuzzy search as shown in the above example. When the genuine record is known as in the above case, a fuzzy search retrieves data that are within a specified distance of the genuine record. The retrieved data then form a fuzzy cluster.", + "p0148": "In a case in which the company does not know which record is the genuine record, for example, both \u201cJohn Smith ox2 6qt\u201d and \u201cJohn Smith ox2 6qy\u201d may be true records, figuring out which records are duplicates of each other, thus creating a fuzzy cluster, cannot be carried out through a simple fuzzy search because there is no a priori guidance on how to group records together. Section explains in detail a few approaches that can be adopted to generate fuzzy clusters in such cases.", + "p0149": "Other Examples of Quantified Differences.", + "p0150": "Distance between two pieces of data is one example of a quantified difference between the two pieces of data. Differences between two pieces of data can be quantified in different ways.", + "p0151": "In some arrangements, a scoring system may be developed to score a matching pair based on the similarity between the pair. The quantified difference between the pair can then be defined as the complement of the normalized matching score.", + "p0152": "In scenarios in which there exists an uncertainty because of a pending outcome of an event, e.g., a lawsuit, the probabilities of a piece of data belonging to one category or another can be used to quantify distances between the piece of data and keys representing the category. The distance between the piece of data and the key representing a category can be defined as the complement of the probability that the piece of data will fall into the category if there are only two categories, or as the conjugate of the probability that the piece of data will fall into the category if there are more.", + "p0153": "Variant Relations and the Variant Join", + "p0154": "Objects from respective data elements in respective datasets to be compared when those pairing data elements in a join operation can be defined as a piece, or a combination of pieces, of data. In a row of a table in a relational database, an object may be the value in a column, a part of a value , or a combination of values from more than one column. In a flat file dataset, consisting of a sequence of records comprised of fields, an object may be the value in one field, a part of one field or a combination of more than one field. In a document, this may be a fragment of text or a combination of disjoint fragments of text.", + "p0155": "Consider a set S of objects . Each object k in S has an associated set, possibly empty, of variant objects, called variants, . The relation", + "p0156": "k\u02dcv", + "p0157": "is read \u201cv is a variant of k\u201d. In some arrangements, two objects are determined to be variants if their score under a function s is below a threshold T", + "p0158": "s < T.", + "p0159": "A distance between objects, say an edit or deletion distance for strings as discussed above, can be used as the basis for constructing a scoring function for comparing words or phrases.", + "p0160": "The variant relation need not be an equivalence relation, that is, being symmetric and having the transitive property , but it sometimes is. The variant relation, even if it is not an equivalence relation, is presumed to be symmetric", + "p0161": "k\u02dcv=>v\u02dck,", + "p0162": "i.e., if v is a variant of k, then k is a variant of v.", + "p0163": "An exact join of two datasets A and B may be defined as a pairing of records , which contain identical objects k A in A, k B in B, such that", + "p0164": "k A =k B .", + "p0165": "The objects k A and k B are called keys.", + "p0166": "A variant inner join is defined in two steps. First a provisional pairing of data elements such as records is made. In one arrangement, k A in A is paired with its variants v Bn in B, k A \u02dcv Bn . The pair of records associated with k A and v Bn is then evaluated, E , to determine which pairs of records to keep. The evaluation operation generally involves comparison of further objects in the paired records beyond the objects used for pairing. In some arrangements, the evaluation operation produces a score which must exceed a match threshold to identify a match. Semi and outer joins are defined by analogy to the exact case with a null value specified for the opposing record when no matching records are found .", + "p0167": "The simplest provisional pairing is given by", + "p0168": "k A \u02dcv Bn ,", + "p0169": "that is, the set of variants of k A in B. This pairing step is supplemented by an evaluation step which determines whether the proposed pairing is to be kept.", + "p0170": "There is a hierarchy of extensions to the variant pairing, k A \u02dcv Bn . The first generalization is to extend the pairs by appending the further pairs , given by", + "p0171": "k A \u02dcv An , v An \u02dcv Bnm .", + "p0172": "That is, k A is paired with the variants in B of the variants of k A in A. When the variant relation is not an equivalence relation, this reaches a larger set of elements in B. Note that this operation is not symmetric: there may be objects v Bnm in B which cannot reach k A . That is, given", + "p0173": "v Bnm =k B , k B \u02dcv Bi , v Bi \u02dcv Aij ,", + "p0174": "no v Aij =k A . This is because none of the variants of k B in B need have k A as a variant\u2014at best, k B is only required to have a variant of k A as one of its variants.", + "p0175": "Further extension to variants of the variants, and so forth, is possible. In particular, extending by the pairs , where", + "p0176": "k A \u02dcv An , v An \u02dcv Bnm , v Bnm \u02dcv Bnmp ,", + "p0177": "is symmetric in the following sense. Given an element k B in B , i.e. k B =v Bnmp for some n, m, p, there is an element v Aijl =k A , where", + "p0178": "k B \u02dcv Bi , v Bi \u02dcv Aij , v Aij \u02dcv Aijl .", + "p0179": "In other words, the same variant matching procedure applied in reverse contains the reversed pair: every object in B reached from an object in A can in turn reach the original object in A through the same procedure.", + "p0180": "The extension to more than two datasets may be defined by joining the datasets pairwise and taking the Cartesian product of the resulting pairs. Thus to join A, B, and C,", + "p0181": "k A \u02dcv Bn ,", + "p0182": "k A \u02dcv Cm ,", + "p0183": "\u21c4 .", + "p0184": "Higher order extensions are obtained by using the higher order extensions defined above in a pairwise fashion. Optionally, in some situations, a variant relation between B and C may be required", + "p0185": "v Bn \u02dcv Cm for some n,m.", + "p0186": "The use of higher order variants may be required to establish this connection directly between B and C .", + "p0187": "As discussed above, one useful source of variant relations is to pair words related by an edit distance. If the edit distance considered between words is limited to one, this admits a certain set of pairings within a dataset as variants. For example, \u201cSmith\u201d would have \u201cSmth\u201d, \u201cSmith2\u201d and \u201cSmyth\u201d as variants. \u201cSmith20\u201d is not a variant of \u201cSmith\u201d, but it is of \u201cSmith2\u201d, hence the variant relation of edit distance one is not transitive.", + "p0188": "The variant join can be used when single words or whole fields can be used as variant keys. For example, searching a dataset can be formulated as a variant join using variant keywords. A query phrase is decomposed into a set of keywords, each of which is matched to its variants in an index of words from the target dataset. The index pairs a word with a record identifier for each record containing the word in a given field in the target dataset. A list of corresponding record identifiers is obtained from the variant match of each keyword with the index, and these lists may be intersected to find records sharing one or more keywords. The list of returned records can be ordered by assigning a score to the combination of matching keywords. This score may take into account the relative frequency of each keyword in the dataset , the relative position of the keywords in the query phrase as compared with their position in the target dataset records , or the absence of words from the query phrase. Keywords can also be associated with other measures of relevance to make scoring more discriminating.", + "p0189": "The variant join can also be used as a lookup on a single word. For example, a customer in one dataset may be identified by firstname, lastname and address. This customer may be sought in a second dataset by lastname with the match to be corroborated by firstname and address. The match procedure is: from the lastname in the source dataset, use the set of variant lastnames in the target dataset to identify and retrieve the set of match candidates. These candidates are further compared on firstname and address to determine if the degree of agreement is sufficient to identify a match.", + "p0190": "For example, suppose the record in the source dataset is", + "p0191": "- Paul,Smith,20 Walker Street", + "p0192": "and the set of matching variants in the target dataset is . The associated records in the target dataset are", + "p0193": "- 1,Paul,Smith,20 Walken St - 2,Robert,Smith,1532 East Grove Ave - 3,P,Smyth,19 Western Ave - 4,Pal,Smithh,20 Walker Street", + "p0194": "The corroboration algorithm might find records 1 and 4 are sufficiently close to be a match. These records might be returned by a lookup or in a variant join .", + "p0195": "Alternatively, perhaps in the source dataset, the original \u201cSmith\u201d has a variant \u201cSmith2\u201d which has a matching record in the target", + "p0196": "- 5,P,Smith20,Walker Street", + "p0197": "\u201cSmith20\u201d is not a direct variant of \u201cSmith\u201d but can be reached from the variant \u201cSmith2\u201d in the source dataset.", + "p0198": "Another use of the variant join is to define superclusters prior to clustering. This will be discussed below after clustering is defined.", + "p0199": "Clusters and Partial Membership", + "p0200": "Many exact-key-based data operations require records be grouped into sets sharing a common key value. These sets are sometimes called \u201ckeygroups\u201d. For example, the rollup operation combines or aggregates data across the records in a keygroup to return a single record. Counts, totals, maximum or minimum values, vectors of values, deduplication to a unique value, etc., can all be computed with a rollup operation. Any operation which summarizes a group of records into a single record may be construed as a rollup operation.", + "p0201": "Data-parallel processing, in which data is segregated into data partitions for independent processing, often relies on key-based partitioning to ensure that all records belonging to the same keygroup are present in the same data partition. Operations like rollup and join depend on this to produce the same result as they would in serial processing.", + "p0202": "The set of keygroups constitute a partitioning of the set of all records into disjoint sets: every record is a member of one and only one keygroup. Clusters generalize the notion of keygroups to partitions involving overlapping sets, where membership is not determined by exact agreement of keys.", + "p0203": "Consider the partitioning of a set S into a collection of possibly overlapping sets , called clusters, of objects k, each with weight w . An object k may be a member of more than one cluster C, and, if so, its cluster membership is said to be ambiguous or partial. The weight w , assigned to k in C, quantifies the \u201cpartial membership\u201d of k in C and is sometimes called the measure of ambiguity. The cluster C may be denoted as the set of pairs C= . If w =0, then k is said to be \u201cnot a member of C\u201d. If w =1, then k is \u201cdefinitely a member of C\u201d. For fixed k, the sum of the weights over C is equal to 1, corresponding to definite membership in S,", + "p0205": "The assignment of weights is associated with a rule R and may be labeled by R. A given set S typically admits more than one partitioning into a collection of clusters and more than one weighting assignment for each object k under different rules. In general, weights associated with different rules cannot be combined.", + "p0206": "The complement of a cluster C= is defined to be the set . In particular, the complement contains, with weight 1, the objects not in C. If the collection of clusters do not span S, or the sum of weights fork over C is not equal to one, the complement of the union of the in S is presumed to be adjoined to the collection.", + "p0207": "Two clusters can be combined into a single cluster, thereby coarsening the partitioning, by summing their weights", + "p0208": "C 1 +C 2 = .", + "p0209": "A cluster can be decomposed into further clusters by reversing this process, allocating the weights for each object among the new clusters, so the sum of new weights equals the original weight. Objects can be removed from a cluster, for example after applying a selection criteria, by subtracting their weight.", + "p0210": "In some situations, when the weights satisfy 0<=w <=1, the weights may admit an interpretation as \u201cthe probability that k is a member of the cluster C in S,\u201d but in general the definition of clusters is non-statistical The possibility of negative weights and weights greater than one is not excluded, but the sum of the weights for fixed k must be one.", + "p0211": "While combinations like C 1 +C 1 can in principle be formed, they correspond to constructing clusters containing multiple copies of objects, as reflected by the value of the maximum possible weight for an object . The condition that the sum of the weights be one for each object assumes that only one copy of each object is present in the set S. If this is not the case, the value of the sum of the weights may be changed accordingly. In general, there is nothing to preclude the total weight from varying by object.", + "p0212": "A cluster with partial membership is similar to the notion of a fuzzy set since it can be described as a set of objects with a membership function assigning weights to each object in the set. However, here the emphasis is not on fuzzy sets in isolation but on clusters as elements of a partitioning. In particular, the weights are a property of the partitioning and not of the object within the cluster in isolation. The weight assigned to an object within a cluster is affected by the possible alternative assignments to other clusters. The focus shifts from the membership function within a cluster to the function allocating membership for an object across clusters.", + "p0213": "Clusters with partial membership arise naturally in a number of situations. In simplest terms, partial membership is a consequence of ambiguity in the assignment of an object to a cluster. If there is an exact key, there is no question to which keygroup an object belongs. If membership is based on a piece or combination of pieces of data which need not agree exactly, membership decisions may not be so clear cut.", + "p0214": "The following are examples of broad data quality issues which can lead to partial membership.", + "p0215": "Data may be inherently ambiguous relative to the clustering rule. Sometimes clusters overlap for the simple reason that their definition does not presume exclusive membership. Consider an employee who works for two different departments in a company. If a list of employees is clustered by department, the employee properly appears in two clusters, as this reflects the true state of affairs. In this case, partial membership can be set to reflect the portion of time the employee works for each department. This in turn reduces the opportunity to draw the false conclusion that there are two distinct employees having the same identifying information in the separate departments.", + "p0216": "Data may be imperfect. Variant words in fields may make identification ambiguous. When assigning addresses to households, the house number on one address record may be 12, yet there is no house number 12 on the street. Instead there are house numbers 1, 2 and 21. A weight of 0.3 on the first two and 0.4 on the last might reflect the slightly greater chance of a transposition error over an insertion error.", + "p0217": "Data may be incomplete. A piece of information necessary to make a decisive assignment to a cluster may be missing. For example, consider the problem of assigning address records to households. Each unique house number, street, city, postcode combination is assigned a unique household number. The clustering algorithm may be tolerant of variant spellings of the street and city names, so it is not necessary for every address be identical to be assigned to the proper household. However, if the house number is missing from an address, there is not sufficient information to make a definitive assignment. In order to preserve as much information as possible, the incomplete record may be partially assigned to each household consistent with the available information. If there were five possible housenumbers, then the weight in each household cluster might be 0.2, reflecting the equal likelihood of each house number. In a different context, consider a dataset consisting of the outstanding debt on accounts labelled by company name. A bank wants to aggregate this data to determine the total outstanding debt associated with each company by country. Among the company names are \u201cACME SERVICES LIMITED \u201d, \u201cACME SERVICES LIMITED \u201d and \u201cACME SERVICES LIMITED\u201d. Each of the first two go into separate clusters, but the third is an equal match to each of the first two and lacks a country identifier. Putting the third company in each of the first two clusters with weight 0.5 reflects the incompleteness of the company information.", + "p0218": "Data or classification may be inherently uncertain. Cluster membership may be based on the outcome of future events. Consider a dataset containing a list of assets and their values. The assets are to be clustered by owner. However a lawsuit is pending on the ownership of a particular asset. Placing it with either possible owner may be a lost bet, yet the asset cannot simply be ignored. Assigning the asset to each owner with a partial membership reflecting the current state of knowledge of the expected outcome of the lawsuit gives the fairest and most informative disposition of the asset consistent with current knowledge.", + "p0219": "Clustering", + "p0220": "Clustering is the act of grouping records into clusters, based on cluster membership criteria. In the exact case, an object from each record is exactly matched with a corresponding object in other records and a cluster, or \u201ckeygroup\u201d, is the set of records sharing a common key. In the fuzzy case, cluster membership may be determined by a variant relation between objects in each record. To avoid having to compare all records in a dataset to each other, a supercluster key is used to divide a whole set into subsets, and cross comparison is restricted to records within a supercluster.", + "p0221": "In many cases, superclusters are defined by exact keys, for example, a postcode. The variant join enables a supercluster to be defined using variant objects. For instance, a supercluster may be defined as the set of records containing all postcodes which are variants of a given poscode. For example, given the UK postcode OX2 6QY, the variant postcodes OX26QY and OX2 6QT are both edit distance one variants, while the latter is itself a valid postcode. Admitting records from each variant postcode as potential matches enables the cluster results to be tolerant of errors in the postcode.", + "p0222": "In another arrangement, superclusters may be formed by taking a fragment of a word from a chosen field in each record and using the variants of this fragment to identify the supercluster. This is appropriate when for two records to be members of the same cluster, they very likely share particular words, but those words need not be direct variants, let alone equal. By considering variant fragments as the supercluster key, records are admitted for which the remainder of the word differ more than accepted as a variant. A more thorough comparison of the full word and other objects in each record is needed to determine cluster membership.", + "p0223": "For example, when comparing German street names, the street Graf von Stauffenberg Strasse may be required to contain Stauffenberg in some form. In sample data, the observation is made that Strasse may be abbreviated and concatenated to the preceding word to give entries like Graf v. Sauffenbergstr. A supercluster defined by edit-distance-two variants of the first five characters of the longest word in each streetname would include both \u201cstauf\u201d and \u201csauff\u201d. Records containing both Stauffenberg and Sauffenbergstr would be included for comparison within the supercluster and a suitable scoring function would assign them to the same cluster. By contrast, a supercluster based on edit distance two variants of the longest word would isolate these two streetnames into separate superclusters, and they could not be clustered together.", + "p0224": "Judicious choice of superclusters is important for the performance and accuracy of clustering methods. For example, if superclusters are too big, many unrewarding comparisons may be made which may cause performance to suffer. Alternatively, if superclusters are too narrow, acceptable matches may be missed, and accuracy may suffer.", + "p0225": "Partial Membership", + "p0226": "Suppose a data entry operator is filling in forms in an application to add new customers to a database. As names are entered in the form, the application validates the entries against a reference lists of names. Using fuzzy search with the deletion algorithm as described above, the application can detect a variant spelling of a name and return a list of alternates from the reference list. Suppose, the operator enters \u201cJame\u201d in the first name field. The application might return the following list of alternates in alphabetical order", + "p0227": "Jaime 250", + "p0228": "James 13359", + "p0229": "Jamie 339", + "p0230": "Jane 9975", + "p0231": "These all differ from Jame by one insertion and/or one deletion and are candidate alternatives.", + "p0232": "To improve the usefulness of the list to the operator, the alternatives can be prioritized using any of a variety of models for determining the measure of ambiguity. Three exemplary ways to quantify the ambiguity are: 1) equipartition, 2) statistical frequency, and 3) error model.", + "p0233": "In the equipartition approach, each alternative is treated as equally likely. Here, the likelihood that Jame is any one of the alternatives is one-fourth. An alphabetical list of alternatives typically indicates an implicit equipartition approach.", + "p0234": "In the statistical frequency approach, a reference set, like the database table itself, is used as a source of observed frequencies for each name. If the list above is sorted in descending order by the count shown, then the most likely correction is James, followed by Jane, etc.", + "p0235": "The third method of error model is based on the observation that certain kinds of errors are more likely than others, depending on the language and the mode of entry, among other things. For keyboard entry by a skilled operator, it may be that a substitution error may be more common than skipping a character or inserting an extra character. Similarly, for an operator recording a spelling given by a customer over the phone, transcription errors involving phonetically similar letter names are likely to be more common than other kinds of errors. In either case, here, Jane would be the most likely correction. To use this method, a model classifying the possible errors and their relative importance can be developed and applied. Such a model could be produced from statistical analysis of the WFS file, introduced in U.S. Application Publication No. 2009/0182728.", + "p0236": "Suppose an application matches records containing customer address, here referred to as query addresses, to a master customer address table to retrieve an existing household key, if a match is found, or to create a new key otherwise. Query addresses may not match addresses in the master customer address table exactly, so a fuzzy match can be used. Furthermore, the query addresses may be incomplete or inaccurate. This means that more than one existing address may be a match to the query address. To quantify the quality of the match, it is useful to have a measure of the ambiguity of the match.", + "p0237": "For example, an address may have no house number while the master customer address file has multiple entries with the same street address . Suppose the query address is Lower Street with a particular town and postcode. A fuzzy search on postcode returns a list of prospective address records, having the same or a variant postcode. The postcode, town, street, and house number fields on the query address are compared with each corresponding field of the prospective addresses and scored, as part of the fuzzy match process. Suppose, in this example, there are two master records which exactly match the street, town and postcode of the query address: 2 Lower Street and 3 Lower Street. Each prospect has equal quality of match with the query record, and this cannot be improved upon with the existing data because the house number is missing on the query record. Under the equipartition measure, there is an equal likelihood that the actual match is to either household.", + "p0238": "Alternatively, suppose the house number is populated but invalid, failing to correspond with any existing address. Suppose the query address is 12 Lower Street, but validation against a reference Postal Address File shows there is no house with that address. The matching addresses in that postcode are 2 Lower Street and 3 Lower Street as above. An error model for address entry might prefer the match of 12 to 2 over a match of 12 to 3. This would give a biased weighting of the likelihood of the match to favor the match with address of 2 Lower Street.", + "p0239": "Finally, if the house number on the query record is populated and a valid postal address, then an error model for address entry could quantify the likelihood that the address is new versus an error of an existing address.", + "p0240": "Quantifying Data Quality", + "p0241": "A measure of ambiguity is also applicable in the wider context of measuring data quality. Businesses and organizations are concerned about the quality of their data, particularly their master data, but currently it is difficult to quantify any but the most obvious data quality problems. Of the short list of data quality problems given above, some data quality measuring systems mainly address one directly: data validity. Data can be exhaustively cataloged and checked for validity against its data type and against a variety of user-defined measures of validity, including lists of valid values.", + "p0242": "Evidence of incomplete entries within fields of records can be inferred from the number of unpopulated entries, but this does not quantify the importance of the missing information. Consider the case of a master customer address list. If the city is missing from a customer address entry, but there is a valid zipcode and street address, does this add any ambiguity? Or, can the address effectively be completed from the information on hand, perhaps using a reference set like a Postal Address File? What if an address is missing a house number? How many houses share the remaining address? Some data quality problems can be fixed with data cleansing , others cannot . A measure of the intrinsic ambiguity present in the data is needed.", + "p0243": "By comparing each address entry to a reference dataset, a measure of the ambiguity in the entry can be computed. An ambiguity report might include the fraction of entries with no ambiguity. For entries having ambiguity, a report might show a histogram plot of the number of entries having K alternatives . There might also be a list of the first N entries with the greatest ambiguity, where N is a user-specified number of entries. One summary statistic quantifying the ambiguity associated with incomplete data for the entire dataset can be constructed from the mean and standard deviation of the number of alternatives per entry.", + "p0244": "If a statistical frequency measure of ambiguity is applied to quantify the likely completions of an address, then interesting measures are: a) histogram plot of number of entries with K alternatives, b) list of N entries with the greatest range of alternatives, with histogram plot of distribution of frequencies of the alternatives, c) count and list of N entries with the strongest association to a single alternative, d) mean and standard deviation of number of alternatives.", + "p0245": "Analogous measures apply to the error model measure of ambiguity.", + "p0246": "The data quality problem of variant entries, where the entry in a field is not the one intended, is similar to both the problem of invalid values and the problem of incomplete information. At some level, saying an entry is not the one intended is to assert that it is invalid for a specific, though not necessarily articulated, validation criterion. Validation of a street name might be defined by comparing the street name to that contained in a reference Postal Address File. Alternatively, in the absence of a reference dataset, validity might be inferred from the relative frequency of associated variant matching entries. If an entry occurs with relatively high frequency and there are no high frequency alternatives, it could be taken as valid. If it occurs with relatively low frequency and there is a single high frequency alternative, it may be taken as invalid. If there are multiple high frequency alternatives, the correction to valid data may be ambiguous and may be quantified similarly to how it would be were the data missing.", + "p0247": "Suppose a field has a value outside an enumerated set of valid values. For example, the gender field is G instead of M or F. An equipartition measure of ambiguity would say there were 2 alternatives for the entry. A frequency measure would still show 2 alternatives but might contain a bias to either M or F. In a simple enumerated case, when there is no variation in the number of alternatives, an ambiguity measure for the dataset is formed from a simple product of the ambiguity measure per entry times the fraction of invalid values.", + "p0248": "When there is variation in the number of alternatives, as there would be if, say, a street name were misspelled, then the measure of ambiguity will help to quantify the ambiguity present in the data. The frequency and error model measures should give the most reliable results for variant spellings. As with incomplete information, the measure of ambiguity ultimately reflects how much the dataset could be improved by cleansing and how much uncertainty would still remain.", + "p0249": "Data entered in the wrong fields could also be quantified in a similar way. Here, there may be the additional ambiguity regarding whether the data placement is in fact wrong. In the case of a surname having a common first name value, it may not be clear whether one ordering over another is correct. Knowing how often there is a common first name in the surname field helps to bound how serious the problem of wrongly ordered names might be. If an accurate reference is available, then a measure of the error rate could be obtained. It may be that out of a dataset of 100,000 entries, there are 500 where there is a surname which is a common first name, but only 25 of these are in fact wrongly ordered. The error rate is then 25/500=1/20. Even in the absence of a measured error rate, knowing that there are only 500/100,000=1/200 entries which are vulnerable to the problem improves one's confidence in that data.", + "p0250": "Other interesting checks for the name fields would be to know when: a) both forename and surname are common first names, b) when forename is a common last name while surname is a common first name, c) when forename is a common last name while lastname is as well. A frequency measure of ambiguity derived from a reference dataset of names could be used to compute a likelihood that the ordering is correct. For example, consider the name David Paul. From the probability that David is a first or a last name, and similarly for Paul, the likelihood that David is a first name can be computed.", + "p0251": "Some reorderings among fields, for example in addresses, are wrong relative to a chosen reference dataset but are not wrong per se because the standard which specifies ordering is either weak or non-existent. Here a measure of ambiguity of a particular address ordering compared with the reference dataset could be used to show that different orderings are not a serious data quality problem because they do not introduce ambiguity of association into the data. This is important input to the decision of where to invest effort to improve data quality.", + "p0252": "To quantify the amount of reference data present in free text fields, one could decompose the free text into words which are used in a fuzzy search against chosen reference datasets. For example, suppose a company expects that product ids are being stored in comment fields of invoices. By using each word in the comment field to do a fuzzy search against the product id table, the number of product ids present in the comment field can be found. More generally, each word could be searched against the WFS file to determine all of the fields in which it or its variants occur, and with what fractional rate. This gives a fuzzy cross-correlation of data in one field against data seen in others.", + "p0253": "The several referential integrity problems can all be quantified. First a corroboration measure can be defined as the fraction of exact key pairs for which the linked data is not in fact correctly linked. Corroboration of this kind can only be made when comparable fields exist on both sides of the link . Typically this is a comparison between similar fields, like customer name and address, that are held in more than one database where a linking key has been established between the databases. This situation arises when comparing data held in a data warehouse with data in the source systems that populate the warehouse. If different source systems hold conflicting information or if they are updated inconsistently, the data warehouse may similarly be in conflict with one or more source systems. Validating a data warehouse for consistency with its sources would provide a new and important check on enterprise data quality.", + "p0254": "A second check is to look for missing and orphaned links, that is links where there is no record on the other side of the link. A fuzzy search can determine whether there is another record which should be a linked or not . A measure of the fraction of each of these conditions is important. If the match to a record on the other side of the link is ambiguous, then a measure of ambiguity can quantify it. This forms the basis of a data cleansing operation to repopulate the links where this can be done uniquely and identifies which links require further investigation because they are ambiguous.", + "p0255": "When two datasets contain related information but no key relation exists between them, a fuzzy search or join between the datasets will find prospective links between the datasets. A measure of the ambiguity of each prospective link will indicate how clean the mapping is between the datasets. This would be very useful for example when combining the master reference data, like customer name and address, of two companies when they merge. Equally, it can be used to merge the reference data of different parts of a business. This would be an important early stage in the arrangement of a master data management solution. Part of the gap analysis in creating a master data management solution is to determine the quality of the alignment between existing reference datasets used by different systems. The initial alignment between the systems is a by-product of this analysis. Having a measure of ambiguity then helps to quantify the additional work it will take to reconcile the systems.", + "p0256": "Clusters and Partial Membership", + "p0257": "Creating Fuzzy Clusters", + "p0258": "As we mentioned before, when there is no a priori guidance on how to coalesce elements into groups, a principle or an algorithm is used to identify groups. This is of practical importance because in real cases, which element should serve as a core element to attract other elements in order to form a group often is not clear. For example, in the above example of the duplicate records in a customer address database, sometimes it is not possible for the company to tell which record is the genuine record. The following discussion proposes a few algorithms that can be used to form fuzzy clusters of records, within which the pieces of data will be regarded as associates of each other.", + "p0259": "In the above mentioned customer address example, there are six records associated with \u201cJohn Smith\u201d: John Smith ox2 6qt; John Smith ox2 6qx; John Smith ox2 6qy; John Smith ox2 7qy; John Smith ox26qt; John Smith ox26qy. Without knowing which record corresponds to a real household/real households, the company may be interested in grouping the above records into two or three clusters with each cluster representing a real household. In this way, the company may be able to reduce mail volumes by reducing mail sent to spurious mailing addresses.", + "p0260": "One algorithm that can be used to create clusters is finding the largest disjoint subsets containing the largest number of elements that are within a specified distance. This approach is explained with reference to FIG. 5 . The steps involved are illustrated in the flow chart. FIG. 5 also uses the above customer address database example to elaborate the algorithm. The results from each step are demonstrated to the right of the corresponding step. In this instance, the specified distance is two operations.", + "p0261": "In referring to the flow chart in FIG. 5 , the first step to create clusters out of the largest disjoint subsets is: for each element, count the variants of that element. As defined above, a variant of an element is an element that is within a specified distance from that particular element. In the customer address example, for record A, there are three records that are within the distance of two operations. For record B, there are two records that are within the distance of two operations. For C, there are four; for D, there is one; for E there are two; and for F there are two.", + "p0262": "Then, select the element with the largest number of variants and make the element and its variants a group labeled by the element. In the customer address database example, record C has the largest number of variants, 4. Record C and its variants form the first cluster.", + "p0263": "Next, from the set of all elements, remove the elements of the largest group. In the customer address example, this leaves only record E.", + "p0264": "Then in the remaining elements, find the element with the largest number of variants. This step will generate a second cluster. In the customer address example, the second cluster has only one element, E, in it.", + "p0265": "Continue until all elements have been grouped in a cluster. In the customer address example, there is no need to go further because every element has found its group.", + "p0266": "This algorithm generates two clusters in the customer address database example, a group that consists of A, B, C, D, F and a group that consists of solely E. The company may treat the records contained in each group as duplicates of each other and consolidate the records to reduce mail volumes.", + "p0267": "Some adjustments may be added to the above algorithm. For example, both records A and F are of the same distance from C and E. In the above algorithm, assigning records A and F as duplicates of C is an artifact of the process and does not necessarily indicate that the records A and F are closer to C than to E.", + "p0268": "One adjustment may be to note the uncertainty on the expressions of the clusters. For example, an expression C 5\u22122 can be used to represent the cluster of C, which includes record C and its variants, with 5 indicating the total number of records and \u22122 the uncertainty. An expression E 1+2 can be used to represent the cluster of E, which includes record E and its variants with 1 indicating the total number of records in that group and 2 the uncertainty. A positive uncertainty of a cluster reflects that there are elements that are grouped elsewhere may belong to this cluster. A negative uncertainty of a cluster reflects that elements in this cluster may belong to another group.", + "p0269": "Another adjustment may be to add A and F to the cluster of E. Thus, group C has records A, B, D, F and group E has records A and F. However, because the records A and F belong to two groups, the total number of records in all of the groups is 8, two more than the true total count of records. To conserve the total count, partial membership may be used.", + "p0270": "A second method of constructing fuzzy clusters is appropriate when clustering data on multi-word fields, like company name, where the variation between records is assessed by scoring phrases rather than single words . Phrase scoring takes into account not only variant spellings of words, but also word order, missing words, and insertions between words that change the detailed word sequence. For example, given a company name Bank of America, the following illustrate four types of name variations that need to be identified and distinguished:", + "p0271": "1) Bank of Amrica", + "p0272": "2) America Bank", + "p0273": "3) Bank America", + "p0274": "4) Bank of South America", + "p0275": "As an example of clustering on a phrase, suppose a bank is attempting to identify all of the accounts that belong to the same legal entity in a master customer list. A legal entity is to be identified by the company name, address and company registration number, if there is one. The principal field used for clustering is company name as it is highly correlated to the legal entity and is always populated. Address is a secondary field used to distinguish companies which accidentally have similar names. Company registration number is expected to be definitive for legal entity identification, but it is insufficiently populated to be used alone.", + "p0276": "The fuzzy clustering operation begins by identifying a super-cluster key which divides the original dataset into smaller subsets, appropriately sized to compare allow comparison of all elements for cluster membership. Records with differing super-cluster key will by construction be in different clusters. For geographically based data, like addresses, the postcode is often an appropriate super-cluster key. Records having a matching variant postcode may be included in the super-cluster. Records with non-matching postcode are expected with high probability to belong to distinct clusters, so to improve performance, they are excluded when computing clusters by introducing the super-cluster key.", + "p0277": "Within each super-cluster, data is sorted in descending order by length of the company name field and in ascending order by company name to present the longest names first to the clustering algorithm in a reproducible order. The first record in a super-cluster group is made the primary record of the first cluster. Each subsequent record, called here the current record, is compared to the primary record of each existing cluster by scoring the company name of the current record against the company name of the primary record of the cluster. If the score is above a suspect match threshold, the cluster is added to a list of suspect clusters for the current record. After comparing the current record with all of the existing primary records, if the suspect list is empty, the current record is made the primary record of a new cluster. If the suspect list has only one entry, and the score is above a match threshold, the current record is added to the cluster on the suspect list. If the suspect list has more than one entry, the company name on the current record is scored against every record in each of the clusters on the suspect list. The current record is added to the cluster where it has the highest score over the match threshold. If there is an equal match over the highest score with records in more than one cluster, the current record is added to the first such cluster. If no score is over the match threshold, the current record becomes the primary record of a new cluster.", + "p0278": "This algorithm has two important features. Because ambiguous matches to multiple clusters are decided in favor of the first matching cluster, some clusters are relatively over-populated with ambiguous members. Also, the order in which records are presented to the algorithm affect specific membership decisions. The initial sort on the length and value of company names is intended to ameliorate this by establishing a fixed order of names. The notion of partial membership, discussed below, gives a richer solution which more accurately reflects the ambiguity of cluster membership.", + "p0279": "An example of ambiguous membership arises in the following set of company names:", + "p0280": "ACME Services Australia Limited", + "p0281": "ACME Services Canada Limited", + "p0282": "ACME Services Limited", + "p0283": "Under a particular scoring, the score of ACME Services Australia Limited with ACME Services Canada Limited is 0.65 which is below the match threshold of 0.75, and the records are placed in separate clusters. ACME Services Limited has an equal score of 0.95 to both clusters. It becomes a member of the ACME Services Australia Limited cluster since it is encountered first.", + "p0284": "Partial Membership", + "p0285": "In the first example in the previous section, records A and F belong to both the clusters C and E. If every appearance of a record in a cluster is counted as one, the total count of records in the clusters C and E is eight, five in the group C and three in the group E , even though there are only six records. In this case, partial memberships can be used to preserve the total count. If a piece of data belongs to more than one group, an appearance of that piece of data is counted as less than one, that is, a fractional number. But the sum of all the appearances of that piece of data should still be one to conserve the total count.", + "p0286": "In some arrangements, a partial membership of an element in a group may be defined to reflect the likelihood of the element belonging to that particular group, using, for example, the measure of ambiguity described above.", + "p0287": "For example, suppose record A has a probability of 40% belonging to group C and a probability of 60% belonging to group E. A partial membership of 0.4 can be assigned to record A in group C and a partial membership of 0.6 can be assigned to record A in group E.", + "p0288": "Similarly, suppose that record F has a probability of 10% belonging to group C and a probability of 90% belonging to group E. A partial membership of 0.1 can be assigned to record F in group C and a partial membership of 0.9 can be assigned to record F in group E.", + "p0289": "With the partial memberships assigned to record A and F, the total count is the sum of the count of group C and the count of group E , which is 6. Therefore, the total count is preserved.", + "p0290": "Since the origin of partial membership is uncertainty over the membership of particular elements, the total membership of each group is only known with a degree of uncertainty, i.e., a margin of error. The total count of each group may be expressed as the sum of the memberships, whole and partial, adjusted by the margin of error. This margin may be indicated by the maximum and minimum bounding values obtained by assuming all uncertain decisions about membership fall either to inclusion or exclusion. These correspond to the worst case scenarios over distribution of the members among clusters. Here, with bounding values, total membership in C would be 3.5 : this is read as saying the expected number of members of C is 3.5 while C has at least 3 members and at most 5 members. Similarly total membership in E would be 2.5 : the expected total membership of E is 2.5, but at least 1 member and at most 3 members.", + "p0291": "The bounding values belonging to different clusters are correlated, though the notation used here does not indicate it. Correlations among different records in the dataset are possible and should be taken into account when computing bounding values. For example, it is sometimes possible to know that A and F are not in the same cluster, without knowing to which cluster they belong.", + "p0292": "In the second example above, the quality of clustering can be enhanced by associating to each cluster all records which match the primary record above the suspect threshold, especially when there is an ambiguous match or suspect match to more than one cluster. The quality of the match should be recorded against each suspect record and quantified with a measure of ambiguity. The partial members would in one arrangement be held separately from the full members, labeled with the measure of their partial membership. For example, the members of a cluster might be listed in descending order of partial membership .", + "p0293": "A rule label should be attached to each record as well which can be used both to link records whose partial memberships were determined together and to identify the rule, event or decision which determined the partial membership allocation. This rule label will be useful when adjusting partial membership when combining records with differing partial membership.", + "p0294": "While, from one perspective, partial membership reflects ambiguity arising from uncertainty of membership, from another perspective, partial membership is simply allocation of membership among multiple clusters, as in the example of an employee working for two departments. In the uncertain case, a change in the state of knowledge is expected to change the membership allocation. Alternatively, the partial membership may simply be accepted as definitive. There is no cost to accepting the fractions as real.", + "p0295": "When partial memberships represent the likelihood of an element belonging to different clusters, a partial membership is always non-negative and the sum of the partial memberships of an element belonging to different clusters should one.", + "p0296": "However, a partial membership may be negative in some arrangements. But the sum of the partial memberships of an object belonging to different clusters must still be constrained to be one.", + "p0297": "In some arrangements, a partial membership of an element can be defined as a function of the distance between the element and the primary or a function of a matching score between the element and the primary. One method of constructing partial membership out of fuzzy scores is through a measure of ambiguity, as described above. Different fuzzy scores reflect different distances between an element and the primary, and therefore different measures of ambiguity. Please note that fuzzy scores reflect the resemblance between variants and the primary, and often are not the same as probabilities.", + "p0298": "Fuzzy Data Operations", + "p0299": "Filtering in the Presence of Partial Membership", + "p0300": "Often it is useful to apply a selection criterion to isolate a subset of records sharing a common property. For example, in a dataset of international records, the records from a particular country may be selected. The selection operation is not considered key-based as the field used in the expression determining selection need not be keys. When records are allowed to have partial membership in multiple clusters, filtering may cause some of the partial members to be dropped. The result is the total membership allocation associated with a record across the selected subset may be less than unity. The explanation for this is that the total allocation measures membership in the selected subset against the alternative of being outside the selected subset.", + "p0301": "Suppose that ACME Services Limited has a 0.5 allocation to the group containing ACME Services Australia Limited and a 0.5 allocation to the group containing ACME Services Canada Limited. The total allocation for ACME Services Limited across the whole dataset is 1.0. If a filter is applied to keep only records associated with Canada, ACME Services Limited will have total allocation 0.5 in the resulting dataset. This indicates that ACME Services Limited has 50% chance of being in the Canada subset against a 50% chance for the alternative of not being in the Canada subset.", + "p0302": "Parallel Partitioning by Key and Partial Membership", + "p0303": "In parallel processing records may be allocated to different processing partitions based on the value of a key . When records are allowed to have ambiguous cluster membership, partitioning may be done on a key associated to each cluster. Under this partitioning scheme, the total allocation within a partition associated to a given record may be less than unity. The interpretation of this is analogous to that with filtering: it measures the allocation of the record to the partition against the alternative of not being in the partition.", + "p0304": "Suppose that ACME Services Limited has a 0.5 allocation to the group containing ACME Services Australia Limited and a 0.5 allocation to the group containing ACME Services Canada Limited. A partition by key operation may allocate the group containing ACME Services Australia Limited to one partition and ACME Services Canada Limited to another partition. The total allocation in the latter partition associated to ACME Services Limited record is 0.5, reflecting its association with the ACME Services Canada Limited cluster, against the alternative of 0.5 that it is in some other partition .", + "p0305": "Parallel versions of the familiar data operations may be defined by their behavior within a single partition, with no communication between the partitions. When the total allocation within a partition for a record is less than unity, this is to be interpreted in the sense defined here.", + "p0306": "Rollup and Partial Membership", + "p0307": "The Rollup operation aggregates or summarizes data from the level of individual records to a group level. In the exact key case, a keygroup is defined as the set of records sharing a common key . In the cluster case, a group is defined as the set of records whose members are determined by a comparison test, with the possibility that one or more records may be members of one or more groups.", + "p0308": "Additive numerical aggregation in cluster groups is done as a weighted aggregate using the allocation measure for the weight. Bounding values are computed by computing the aggregation for the alternatives where all records having a partial allocation to the set are either included in the set or are excluded from the set. The following list of records is a cluster group based on company name", + "p0309": "cluster_key alloc_measure company_name count c1 1.0 ACME Services Australia Limited 80 c1 1.0 ACME Services Limited 60 c1 0.5 ACME Services Limited 100", + "p0310": "The rollup to determine the total count in the cluster is the weighted sum", + "p0311": "80*1.0+60*1.0+100*0.5=190", + "p0312": "with the bounding values", + "p0313": "80*1.0+60*1.0+100*0.0=140,", + "p0314": "80*1.0+60*1.0+100*1.0=240.", + "p0315": "The result for the total count in the cluster group may be expressed as 190 .", + "p0316": "Non-additive summarization is done by considering the extreme cases where records having partial allocations are either included or excluded from the set. The allocation measure can often be used to assign a confidence to the result obtained by including partial members. As an example, records may be sorted within a cluster group on some secondary key, and the rollup may determine which record is first in the sort order. The following list sorts the previous list of records in descending order on count.", + "p0317": "cluster_key alloc_measure company_name count c1 0.5 ACME Services Limited 100 c1 1.0 ACME Services Australia Limited 80 c1 1.0 ACME Services Limited 60", + "p0318": "The rollup to determine the first record in the cluster group in this sort order gives the bounding results:", + "p0319": "- c1 0.5 ACME Services Limited 100 - c1 1.0 ACME Services Australia Limited 80 The allocation measure of 0.5 can be provided with the inclusive result to indicate confidence associated with the inclusive result. In this example, the exclusive result can be considered a worst case result: the maximum is at least 80.", + "p0320": "The rollup operation in the presence of partial membership may be conducted in parallel. To see this, consider first additive numerical aggregates. This is a weighted sum. Such a sum can be broken into partial sums which are computed separately and then combined. Each partial sum is a weighted sum which may be computed in its own parallel partition. This is the parallelization of the additive rollup.", + "p0321": "Computation of the exclusive bound of both additive and non-additive rollups is parallelizable because by definition all partial members are excluded. Thus the computation reduces to an ordinary rollup which is parallelizable .", + "p0322": "Computation of the inclusive bound is parallelizable under certain conditions to prevent double-counting of inclusions. For rollups within a cluster, each partial member only occurs once within the cluster. Therefore the inclusive bound can be computed as a sum of partial sums within a cluster without double counting any members.", + "p0323": "If a rollup is performed across clusters, there may be contributions from the same record appearing in different clusters. This is okay for additive measures because the weights associated with each instance of the record add to give a new overall weight. For the inclusive bound however, each record must only be included once. Generally this requires keeping track in some way of which records occur, and this operation is not parallelizable.", + "p0324": "If however individual records are identified by a key, say rec_key, prior to clustering, and the data is parallel partitioned on rec_key, then all records having the same rec_key will occur in the same partition. Rollup within this partition can properly compute the inclusive bound since all relevant records are present, even if done across clusters. The inclusive bounds across partitions can then be safely combined across partitions because no individual record has instances on more than one partition so there is no possibility of double-counting.", + "p0325": "Search", + "p0326": "In exact data operations, exact keys are used. For example, in a search operation, a key is used and all the records that exactly match that key are retrieved. In fuzzy data operations, fuzzy keys are used.", + "p0327": "In some arrangements, fuzzy operations are carried out as a series of exact data operations, as illustrated below using search operation as an example.", + "p0328": "In the above customer address database example, the company is interested in finding all the pending mail packages that have been sent to a person named John Smith. A search operation can be used for that purpose. The search operation can be carried out using the two-part key \u201cJohn Smith;ox2 6qt\u201d with the zip code \u201cox2 6qt\u201d being the correct zip code associated with John Smith. This exact search operation, however, will not retrieve those pending mail packages that were sent to John Smith but were mistakenly associated with key \u201cJohn Smith;ox2 6qx\u201d or \u201cJohn Smith;ox26qt\u201d due to typographical errors made by the mail clerk when making the entries.", + "p0329": "To overcome this limitation, a fuzzy search, which is a search associated with a fuzzy key, can be used. A fuzzy key is one of a set of keys that includes a primary key plus all variants that fall within a specified distance of that key. In the above customer address example, a fuzzy key used in the pending mail package search may be defined to include the primary key, \u201cJohn Smith;ox2 6qt\u201d plus all the variants that fall within a distance of two operations.", + "p0330": "In some arrangements, the fuzzy search operation performed on the fuzzy key can be carried out in the following fashion. In step one, perform an exact search on the primary key \u201cJohn Smith ox2 6qt\u201d. Then, in steps two to five, perform four exact searches on the variants that are part of the fuzzy key. In the final step, step six, combine the results retrieved from the above steps one to five. The combined result is the result of the fuzzy search using the fuzzy key.", + "p0331": "When searching on a multi-word field, like a company name or address, it may not be possible to determine the set of variants of that field in advance for the previous procedure to be used directly. Two alternative strategies may be employed. Suppose that a search is being made for all account records associated with the company ACME Services Ltd at 2 Plater Drive, Oxford. In the first strategy, a single-word field, for example, the zip code ox2 6qt, is used as a search key. All records having the key \u201cox2 6qt\u201d or one of its variants within a fixed distance are retrieved by an exact search as \u201cprospective matches\u201d or \u201cprospects.\u201d The company name and address of each prospect are separately scored against the query company name and address.", + "p0332": "Addresses are typically compared initially by concatenating all fields constituting the address into a single string. This has several advantages. First, it puts addresses from different sources into a simple common format. Since the phrase scoring functions are tolerant of missing words or changes in word order, it does not matter that the original fields holding the elements of the source addresses may have been incompletely or inconsistently populated. Equally, the common format is reached without having to parse the address into standard address elements. Parsing typically requires accurate reference data, like a Postal Address File, which may not be available, particularly in foreign countries. Also parsing is relatively expensive computationally, so avoiding it makes address comparison more better performing. If comparison of the concatenated address fields is inconclusive, parsing may be used as a backup attempt to refine the scoring.", + "p0333": "The resulting company name and address scores are compared to match and suspect thresholds. A score exceeding the match threshold indicates the two compared phrases are sufficiently similar to constitute a match. A score exceeding the suspect threshold indicates the phrases are similar but are not sufficiently close to determine a match with confidence. A score for whether the entire record is a match is obtained by combining the scores for individual fields or combinations of fields . Users specify criteria which determine which information must agree and how closely. For example, the city field may be allowed to disagree if the postcodes agree.", + "p0334": "A second strategy is to select words from one or more fields and to use these as fuzzy search keywords. Such words can be chosen as the leading words in a field or on the basis of their significance. Significance is computed from the negative log of the ratio of the number of occurrences of a word or one of its variants in a field to the number of times that field is populated.", + "p0335": "Once the fuzzy search keywords are chosen, each word is looked up in an index of source records by word. This returns a list of indexes of source records which contain the chosen word in a given field. Suppose the search keywords are \u201cLower\u201d, \u201cIslip\u201d sought in any address field of the Postal Address File. The word \u201cLower\u201d might occur in the organization, the street name and the town fields. \u201cIslip\u201d may only occur as a town. Searching on each word gives a list of indexes of records in which the word is found. Intersecting the two lists gives the set of records that contain both words . These form a set of prospect records. Because the prospect records are known to contain a certain number of the search words, they have been effectively pre-qualified. The more search words required to be held in common, the higher the score between the fields is likely to be.", + "p0336": "After applying a filter to only keep, say, records that contain two or more search words, the full records are retrieved and scored against one another. The resulting scores are sorted in descending order. An ambiguity measure could be computed for the set of matches and this would increase the information characterizing the quality of match of the query against the reference.", + "p0337": "Fuzzy Join", + "p0338": "A fuzzy join is similar to a fuzzy search except that instead of using a lookup to retrieve the records, a join is done instead where the entire reference dataset is read and processed against the query dataset. This can be useful for both performance and control. If a reference dataset is too large to fit in memory as an ordinary lookup, it may instead be accessed as a loadable lookup which is held on disk . As each search term is processed, the appropriate pages of the lookup table are accessed from disk. If a sufficiently large fraction of the reference dataset needs to be accessed, it works out to be more efficient to read the entire reference dataset in sorted order in a single pass than proceed with a random-access search. This reduces repeated disk accesses performed by the search process.", + "p0339": "From the control perspective, if a search requires access to multiple reference datasets, a join may be a more convenient way to bring the data from the reference datasets together for scoring. Suppose the query contains customer name and address in a single entry, but the reference data holds customer name and address is separate tables. There may be a third linking table connecting customer name and address. If the target reference data is held in different tables, it is not possible to compare directly the record indexes returned by searching the index file for the search keywords because the indexes refer to records in different datasets. If a key is assigned to the query, then separate searches can be made against each dataset and a join on the query-key will combine the results of these searches and allow the prospects to be fetched and scored.", + "p0340": "If a measure of ambiguity is computed as part of the output of a fuzzy join, then it is possible for multiple matches to result from a join operation, each with a partial membership. Suppose for example, that an address with no house number is joined against the Postal Address File to pick up the name of the organization that the postal address file has on record at that address. Three addresses in the Postal Address File on the appropriate street do not have an organization and can therefore be combined for the purposes of identifying the organization. Two distinct organizations ACME Ltd and Standard Corp. are present at other matching addresses. An equipartition measure of ambiguity would count the number of equivalent occurrence. The output of the join would initially be five records, one for each matching address. A subsequent rollup to organization at the address would lead to an equipartition measure of ambiguity showing the likelihood the organization is either blank , ACME Ltd , Standard Corp. . This result could then be normalized to three records, each with a distinct organization and an associated partial membership.", + "p0341": "Query: join on address to pick up organization", + "p0342": "Query address: Lower Street, ox2 6qt", + "p0343": "Organization Address \u2014 2 Lower Street, ox2 6qt \u2014 3 Lower Street, ox2 6qt ACME Ltd 4 Loower St, ox2 6qt \u2014 5 Lower St, ox2 6qy Standard Corp. 8 Lower St., ox2 6qt", + "p0344": "Result:", + "p0345": "Allocation Partial Organization Address key membership \u2014 Lower Street, ox2 6qt a1 0.6 ACME Ltd Lower Street, ox2 6qt a1 0.2 Standard Corp. Lower Street, ox2 6qt a1 0.2", + "p0346": "Sort and Partial Membership", + "p0347": "A sort which orders records when membership is partial is straightforward to define. For records with ambiguous association, a record for each alternative is created, together with its allocation . Suppose the reference records in the previous example are sorted with the output of the join against the query record. The rule is that partial membership is sorted after full membership in descending order of membership. Subsort on other fields is applied after partial membership is applied. Thus partial membership against one field takes precedence to later keys. This is to preserve the principle that the application of an additional sort field, sorts records within the established order but without changing the higher level order.", + "p0348": "Sort :", + "p0349": "\u2014 2 Lower Street, ox2 6qt \u2014 1.0 \u2014 3 Lower Street, ox2 6qt \u2014 1.0 \u2014 5 Lower St, ox2 6qy \u2014 1.0 \u2014 Lower Street, ox2 6qt a1 0.6 ACME Ltd 4 Loower St, ox2 6qt ACME Ltd Lower Street, ox2 6qt a1 0.2 Standard Corp. 8 Lower St., ox2 6qt Standard Corp. Lower Street, ox2 6qt a1 0.2", + "p0350": "Having applied a fuzzy sort of this kind, a fuzzy rollup to organization can be made based on the organization name without having to store temporary results until all of the data is seen. This is one of the principle uses of sorted data: so that the rollup operation can complete as each key group completes.", + "p0351": "The fuzzy merge operation is analogous to the fuzzy sort. It simply applies the ordering rule to each record on its inputs to determine which record is next in the sort order. Suppose the above dataset is merged with the following one", + "p0352": "ACME Ltd Lower Street, ox2 6qt a2 0.9 Standard Corp. Lower Street, ox2 6qt a2 0.1", + "p0353": "The merged data is", + "p0354": "Merge :", + "p0355": "\u2014 2 Lower Street, ox2 6qt \u2014 1.0 \u2014 3 Lower Street, ox2 6qt \u2014 1.0 \u2014 5 Lower St, ox2 6qy \u2014 1.0 \u2014 Lower Street, ox2 6qt a1 0.6 ACME Ltd 4 Lower St, ox2 6qt ACME Ltd Lower Street, ox2 6qt a2 0.9 ACME Ltd Lower Street, ox2 6qt a1 0.2 Standard Corp. 8 Lower St., ox2 6qt Standard Corp. Lower Street, ox2 6qt a1 0.2 Standard Corp. Lower Street, ox2 6qt a2 0.1", + "p0356": "Usefulness of Fuzzy Data Operations: Addressing Mistakes and Uncertainties and Preserve Accounting Integrity.", + "p0357": "As demonstrated in the above fuzzy search example, fuzzy search operations can retrieve records that conventional searches using exact keys will miss, for instance, those records that contain typographic errors.", + "p0358": "Also as mentioned above, when a classification of a piece of data depends on a pending outcome, clustering or partial membership can be used to accurately capture the uncertainty. At an immediate level, clustering or partial membership may be viewed as equivalent to a combination or a series of discrete non-fuzzy operations. But a few steps away, clustering or partial membership will allow better handling or prediction. In the above example in which an organization's annual budget depends on a pending court decision regarding whether the organization qualifies as a charitable organization or not, an annual budget can be prepared based on the probabilities of a court decision coming out favorably or adversely.", + "p0359": "More specifically, an annual budget may be reserved as following:", + "p0361": "Using the above formula to calculate the reserved tax payment presents a better overall picture of the financial status of the organization and facilitates risk assessments by the upper management. It also gives downstream applications a more reliable figure to rely on, allowing, for example, market analysts to better predict the financial prospects of the organization.", + "p0362": "Partial memberships are also useful in preserving accounting integrity. For example, in ABC Corp's human resource database, we see that John Smith's medical expenses will not be double counted if his memberships in the marketing department and R&D department are 0.5 each.", + "p0363": "The approach described above can be implemented using software for execution on a computer. For instance, the software forms procedures in one or more computer programs that execute on one or more programmed or programmable computer systems each including at least one processor, at least one data storage system , at least one input device or port, and at least one output device or port. The software may form one or more modules of a larger program, for example, that provides other services related to the design and configuration of computation graphs. The nodes and elements of the graph can be implemented as data structures stored in a computer readable medium or other organized data conforming to a data model stored in a data repository.", + "p0364": "The software may be provided on a storage medium, such as a CD-ROM, readable by a general or special purpose programmable computer or delivered over a communication medium of a network to the computer where it is executed. All of the functions may be performed on a special purpose computer, or using special-purpose hardware, such as coprocessors. The software may be implemented in a distributed manner in which different parts of the computation specified by the software are performed by different computers. Each such computer program is preferably stored on or downloaded to a storage media or device readable by a general or special purpose programmable computer, for configuring and operating the computer when the storage media or device is read by the computer system to perform the procedures described herein. The inventive system may also be considered to be implemented as a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer system to operate in a specific and predefined manner to perform the functions described herein.", + "p0365": "A number of embodiments of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. For example, some of the steps described above may be order independent, and thus can be performed in an order different from that described.", + "p0366": "It is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the appended claims. For example, a number of the function steps described above may be performed in a different order without substantially affecting overall processing. Other embodiments are within the scope of the following claims.", + "features": { + "F1": "A method implemented by a data processing system for identifying matches between a first data element with a plurality of fields and one or more second data elements stored in a data storage system.", + "F2": "Accessing from a hardware storage device the first data element with the plurality of fields, with one or more values of the fields representing the key for the first data element, and including comparison fields used to corroborate variant matches.", + "F3": "Accessing from a hardware storage device the one or more second data elements, including search fields specifying search terms or their variants, and comparison fields used to corroborate variant matches.", + "F4": "Accessing one or more variants of the key represented by values of the fields of the first data element, with a variant specified in accordance with a variant relation for the key.", + "F5": "Determining variant matches between variants of the key represented by values of the fields of the first data element and values of search fields of the second data elements.", + "F6": "Corroborating variant matches of variants of the key based on a comparison of comparison fields of the second data elements to comparison fields in the first data element, with a comparison field being different from a search field.", + "F7": "Joining the one or more second data elements with the first data element based on corroborating variant matches.", + "F8": "Returning the one or more second data elements based on corroborating variant matches.", + "F9": "Forming clusters of data elements with each element being a member of at least one cluster, with clusters including second data elements and the first data element.", + "F10": "At least one data element is associated with two or more clusters, with memberships of the element belonging to respective clusters represented by a measure of ambiguity.", + "F11": "Each value of the measure of ambiguity representing a membership of a data element in clusters is between zero and one.", + "F12": "Values of the measure of ambiguity are related to the likelihood of at least one data element belonging to clusters.", + "F13": "Values of the measure of ambiguity are established based on a function representing relationships between data elements and clusters.", + "F14": "Relationships represented by the function are related to the likelihood of a data element belonging to respective clusters.", + "F15": "Performing a rollup operation that calculates a weighted subtotal of a quantity within a first cluster, summing products of quantity values associated with data elements and a measure of ambiguity in the cluster.", + "F16": "Calculating an exclusive subtotal of the quantity by excluding data elements in the first cluster associated with multiple clusters and an inclusive subtotal by including them.", + "F17": "Determining membership of a given data element in a cluster based on values of comparison fields of the element.", + "F18": "Creating the key from values of fields in the first data element.", + "F19": "The hardware storage device includes a volatile or non-volatile hardware storage device.", + "F20": "A system for identifying matches between a first data element with fields and second data elements, including processors and storage devices executing operations to perform access, determine matches, and join or return elements.", + "F21": "The system's operations include joining second data elements with a first data element based on corroboration.", + "F22": "The system's operations include returning second data elements based on corroboration.", + "F23": "The system's operations include forming clusters of data elements with elements as members of clusters, with clusters including second and first data elements.", + "F24": "The system includes a volatile or non-volatile hardware storage device.", + "F25": "Machine-readable hardware storage devices store instructions executable by processors for performing operations including accessing, matching, corroborating, and storing data element relationships and clusters." + } + } + }, + { + "FAN": "75800075", + "CPC": [ + "A41D-001/00/2", + "A41D-015/00", + "A41D-027/08", + "A41D-031/00/05", + "D01D-005/30", + "D01F-008/00", + "D03D-001/00/88", + "D03D-015/54", + "D03D-015/547", + "D03D-015/60", + "D03D-015/67", + "D10B-2401/18", + "D10B-2401/20", + "G02F-001/13/13", + "G02F-001/1333", + "G02F-001/1334", + "G02F-001/1343/09", + "G02F-001/155", + "G02F-001/167", + "G02F-001/16757", + "G02F-001/1676", + "G02F-2001/1555", + "G02F-2202/022", + "G06F-003/0482", + "G06F-003/0484/2", + "G06F-016/00", + "G06F-017/30", + "G06Q-030/02/51", + "G06Q-030/06", + "G06Q-030/06/21", + "G06Q-050/04", + "G09G-005/00/3", + "G09G-005/00/6", + "G09G-2320/041", + "G09G-2360/144", + "H01L-051/52/87", + "H04L-067/12", + "H04N-005/44", + "H04N-005/44/03", + "H04N-021/422/04" + ], + "Content": { + "title": "Smart garment, user terminal, system including same, and method of changing design of smart garment", + "pa01": "The disclosed embodiments provide a smart garment on which implemented designs can be changed in terms of color, image, text, etc. Also, a system is provided that comprises: a server for providing various designs to be implemented on a smart garment; and a user terminal that can change the design of the smart garment by receiving various designs from the server and transmitting same to the smart garment. The system according to the disclosed embodiments comprises: a server including a design database for a smart garment; a user terminal for downloading a design for the smart garment from the server; and the smart garment on which the design transmitted from the user terminal is implemented.", + "c-en-0001": "A smart garment comprising: a first material comprising a first electrode and a second electrode; a second material comprising a third electrode and a fourth electrode; and a connection module comprising a first ground terminal configured to ground one of the first electrode and the second electrode in response to a signal transmitted by a user terminal, a first signal terminal applying a voltage to another one of the first electrode and the second electrode, a second ground terminal configured to ground one of the third electrode and the fourth electrode, and a second signal terminal applying a voltage to another one of the third electrode and the fourth electrode, wherein the first material and the second material comprises cholesteric liquid crystal fibers including a cholesteric liquid crystal, wherein the cholesteric liquid crystal comprises: a pixel including a red sub-cell configured to reflect red light, a green sub-cell configured to reflect green light, and a blue sub-cell configured to reflect blue light; and a grid provided to separate the red sub-cell, the green sub-cell, and the blue sub-cell, and formed from textile fibers, and wherein the pixel is provided so that the red sub-cell, the green sub-cell, and the blue sub-cell are formed to be coplanar or the red sub-cell, the green sub-cell, and the blue sub-cell are formed to have a stacked structure.", + "c-en-0002": "The smart garment of claim 1 , wherein each of the cholesteric liquid crystal fibers further comprises a first cholesteric liquid crystal fiber electrode provided at one side of the cholesteric liquid crystal, a second cholesteric liquid crystal fiber electrode provided at a side opposite the first cholesteric liquid crystal fiber electrode, an insulator provided between the first cholesteric liquid crystal fiber electrode and the second cholesteric liquid crystal fiber electrode, and a protective layer provided to cover the first cholesteric liquid crystal fiber electrode, the second cholesteric liquid crystal fiber electrode, and the insulator.", + "c-en-0003": "The smart garment of claim 2 , further comprising a connection portion provided to connect the cholesteric liquid crystal fibers so as to extend the cholesteric liquid crystal fibers, and wherein the connection portion comprises a first connection electrode provided to connect the first cholesteric liquid crystal fiber electrodes of the cholesteric liquid crystal fibers, a second connection electrode provided to connect the second cholesteric liquid crystal fiber electrodes of the cholesteric liquid crystal fibers, and an insulator provided between the first connection electrode and the second connection electrode.", + "c-en-0004": "The smart garment of claim 2 , the cholesteric liquid crystal comprises a partition wall provided to maintain a shape of the cholesteric liquid crystal fibers.", + "p0001": "CROSS-REFERENCE TO RELATED APPLICATIONS", + "p0002": "This application is a continuation of U.S. patent application Ser. No. 15/765,484 filed on Apr. 2, 2018, which is a 371 of International Application No. PCT/KR2016/003403 filed Apr. 1, 2016, which claims priority to Korean Patent Application No. 10-2015-0139231 filed Oct. 2, 2015, the disclosures of which are herein incorporated by reference in their entirety.", + "p0003": "BACKGROUND", + "p0004": "1. Field", + "p0005": "The disclosed technology relates to a garment capable of changing a design, such as a color, an image, or the like.", + "p0006": "2. Description of Related Art", + "p0007": "Recently, various wearable devices equipped with an Internet function are being developed. Along with this trend, the technology of Internet of Things is continuously developing.", + "p0008": "Research has been continuously conducted to develop garments that perform various functions by introducing IoT technology to people's clothing in addition to electrical appliances.", + "p0009": "SUMMARY", + "p0010": "Disclosed embodiments provide a smart garment capable of changing design, such as a color, an image, or text, which are implemented on the garment.", + "p0011": "In addition, disclosed embodiments provide a system including a server which provides various designs to be implemented on a smart garment and a user terminal which receives the various designs from the server and transmits the designs to the smart garment to change a design of the smart garment.", + "p0012": "A system according to the disclosed embodiment includes: a smart garment provided to allow a design including a color, an image, text, and combinations thereof to be changed; a server including a design database with respect to the smart garment; a provider terminal provided to upload a design of the smart garment to the server; and a user terminal provided to download the design of the smart garment from the server and transmit a signal corresponding to the design of the smart garment to the smart garment to change the design of the smart garment.", + "p0013": "Further, the smart garment may include a fabric including color-changing fibers and textile fibers, and a connection module provided to receive the signal transmitted by the user terminal.", + "p0014": "Further, the color-changing fiber may include at least one of a cholesteric liquid crystal fiber, an electronic ink fiber, an electrochromic fiber, and an electroluminescent fiber.", + "p0015": "Further, the user terminal may be paired with the smart garment and transmit a design selected by a user to the paired smart garment.", + "p0016": "Further, the user terminal may display a garment display area provided to allow the smart garment to be displayed, a color selection area provided to allow a color of the smart garment displayed in the garment display area to be selected, an image selection area provided to allow an image, which will be implemented on the smart garment displayed in the garment display area, to be selected, and a text input area provided to allow text, which will be implemented on the smart garment displayed in the garment display area, to be input.", + "p0017": "A user terminal according to the disclosed embodiment includes: a communicator configured to communicate with a server and a smart garment; a display configured to display a design of the smart garment transmitted from the server and received by the communicator; and a processor configured to transmit a signal including selected design information to the smart garment through the communicator when the design of the smart garment displayed on the display is selected.", + "p0018": "Further, the display may display a search button provided to receive a command for searching for a smart garment, a garment display area in which a found smart garment is displayed when the command is input via the search button, and a pairing button provided to receive a command for pairing with a garment selected among garments displayed on the garment display area.", + "p0019": "Further, when the selected smart garment is paired with the user terminal through the pairing button, the display may display the paired smart garment in the garment display area.", + "p0020": "Further, when the command is input via the search button, the processor may search for smart garments around the user terminal and be paired with a smart garment selected by a user among the found smart garments when the command is input via the pairing button.", + "p0021": "Further, the display may display a garment display area in which the smart garment is displayed, a color selection area provided to allow a color of the smart garment displayed in the garment display area to be selected, an image selection area provided to allow an image, which will be implemented on the smart garment displayed in the image display area, to be selected, and a text input area provided to allow text, which will be implemented on the smart garment displayed in the garment display area, to be input.", + "p0022": "Further, the display may display the color selection area below the garment display area, display the image selection area below the color selection area, and display the text input area below the image selection area.", + "p0023": "Further, when the smart garment displayed in the garment display area is paired with the user terminal, the display may display an object representing that the smart garment displayed in the garment display area is paired with the user terminal.", + "p0024": "Further, when the smart garment displayed in the garment display area is changed, the display may change a color and a type of an image displayed in the color selection area and the image selection area to a color and an image which are associated with the changed smart garment, and display the changed color and image.", + "p0025": "Further, the display may display at least one of an image received from the server, an image stored in the user terminal, and an object for driving a camera of the user terminal in the image selection area.", + "p0026": "Further, when the object is selected and capturing of the camera is performed, the display may apply an image captured by the camera to the smart garment displayed in the garment display area.", + "p0027": "Further, the processor may transmit the captured image to the smart garment.", + "p0028": "Further, when the image displayed in the image selection area is selected, the processor may transmit the selected image to the smart garment and the display may display the selected image on the smart garment displayed in the garment display area.", + "p0029": "Further, when the color displayed in the color selection area is selected, the processor may transmit the selected color to the smart garment and the display may apply the selected color to the smart garment displayed in the garment display area.", + "p0030": "Further, when text is input to the text input area, the processor may transmit the input text to the smart garment and the display may display the input text on the smart garment displayed in the garment display area.", + "p0031": "Further, the processor may generate a design, which will be implemented on the smart garment on the basis of location information or weather information, and transmit the generated design to the smart garment through the communicator.", + "p0032": "Further, the processor may generate a design based on surrounding information of the user terminal and transmit the generated design to the smart garment.", + "p0033": "A smart garment according to the disclosed embodiment includes: a first material including a first electrode and a second electrode; a second material including a third electrode and a fourth electrode; and a connection module including a first ground terminal configured to ground any one of the first electrode and the second electrode in response to a signal transmitted by a user terminal, a first signal terminal applying a voltage to the remaining one of the first electrode and the second electrode, a second ground terminal configured to ground any one of the third electrode and the fourth electrode, and a second signal terminal applying a voltage to the remaining one of the third electrode and the fourth electrode.", + "p0034": "Further, the connection module may include a connector provided to be connected to the user terminal using a wire.", + "p0035": "Further, the connection module may include a communicator provided to communicate with the user terminal through wired/wireless communication.", + "p0036": "Further, when the communicator receives a signal transmitted by the user terminal, the connection module may include at least one driver provided to apply a voltage to the smart garment in response to the received signal.", + "p0037": "Further, when the communicator receives the signal transmitted by the user terminal, the connection module may include a processor provided to generate a control signal for controlling the at least one driver.", + "p0038": "Further, the connection module may include a memory provided to store information on the smart garment and, when the communicator receives the signal transmitted by the user terminal, provided to store information included in the received signal.", + "p0039": "Further, the connection module may include a battery provided to supply power.", + "p0040": "Further, the first material and the second material may include cholesteric liquid crystal fibers.", + "p0041": "Further, the cholesteric liquid crystal fiber may include a cholesteric liquid crystal, a first electrode provided at one side of the cholesteric liquid crystal, a second electrode provided at a side opposite the first electrode, an insulator provided between the first electrode and the second electrode, and a protective layer provided to cover the first electrode, the second electrode, and the insulator.", + "p0042": "Further, the cholesteric liquid crystal fiber may further include a connection portion provided to connect the cholesteric liquid crystal fibers so as to extend the cholesteric liquid crystal fiber.", + "p0043": "Further, the connection portion may include a first connection electrode provided to connect the first electrodes of the cholesteric liquid crystal fibers, a second connection electrode provided to connect the second electrodes of the cholesteric liquid crystal fibers, and an insulator provided between the first connection electrode and the second connection electrode.", + "p0044": "Further, the cholesteric liquid crystal may include a partition wall provided to maintain a shape of the cholesteric liquid crystal fiber.", + "p0045": "Further, a cross section of the cholesteric liquid crystal fiber may be provided in any one of a circular shape and a polygonal shape.", + "p0046": "Further, the cholesteric liquid crystal may include a pixel including at least one of a red sub-cell configured to reflect red light, a green sub-cell configured to reflect green light, and a blue sub-cell configured to reflect blue light.", + "p0047": "Further, the pixel may be provided so that the red sub-cell, the green sub-cell, and the blue sub-cell are formed to be coplanar.", + "p0048": "Further, the pixel may be provided so that the red sub-cell, the green sub-cell, and the blue sub-cell are formed to have a stacked structure.", + "p0049": "Further, the cholesteric liquid crystal may include a grid provided to separates the red sub-cell, the green sub-cell, and the blue sub-cell.", + "p0050": "Further, the grid may include textile fibers.", + "p0051": "Further, the first material and the second material may include electronic ink fibers.", + "p0052": "Further, the electronic ink fiber may include a plurality of electronic ink capsules including an electronic ink, a first electrode provided at one side of the electronic ink capsules, a second electrode provided at a side opposite the first electrode, an insulator provided between the first electrode and the second electrode, and a protective layer provided to cover the first electrode, the second electrode, and the insulator.", + "p0053": "Further, the electronic ink capsule may include at least one electronic ink among a red electronic ink configured to reflect red light, a green electronic ink configured to reflect green light, and a blue electronic ink configured to reflect blue light, a black electronic ink configured to absorb light, and a white electronic ink configured to reflect light.", + "p0054": "Further, the first material and the second material may include electrochromic fibers.", + "p0055": "Further, the electrochromic fiber may include a first electrode, a counter electrode provided outside the first electrode, an electrolyte provided outside the counter electrode, a working electrode provided outside the electrolyte, a second electrode provided outside the working electrode, and a protective layer provided to cover the second electrode.", + "p0056": "Further, at least one of the counter electrode and the working electrode may include a color-changing material.", + "p0057": "Further, the counter electrode may include a first color-changing material that changes color as the first color-changing material is oxidized.", + "p0058": "Further, the working electrode may include a second color-changing material that changes color as the second color-changing material is reduced.", + "p0059": "Further, the first material and the second material may include electroluminescent fibers.", + "p0060": "Further, the electroluminescent fiber may include a first electrode, a light emitter provided outside the first electrode, a second electrode provided outside the light emitter, and a protective layer provided to cover the second electrode.", + "p0061": "Further, the electroluminescent fiber may include a first electrode, a light emitter provided outside the first electrode, at least three second electrodes provided outside the light emitter and provided not in contact with each other, a red color filter, a green color filter, and a blue color filter which are each provided outside a corresponding one of the at least three second electrodes, and a protective layer provided to cover the red color filter, the green color filter, and the blue color filter.", + "p0062": "Further, the electroluminescent fiber may include a first electrode, a red light emitter, a green light emitter, and a blue light emitter provided outside the first electrode, a second electrode provided outside each of the red light emitter, the green light emitter, and the blue light emitter, and a protective layer provided to cover the second electrode.", + "p0063": "Further, the electroluminescent fiber may include a core fiber, at least three first electrodes provided outside the core fiber and provided not in contact with each other, a red light emitter, a green light emitter, and blue light emitter each provided outside a corresponding one of the at least three first electrodes, a second electrode provided outside each of the red light emitter, the green light emitter, and the blue light emitter, and a protective layer provided to cover the second electrode.", + "p0064": "A method of changing a design of a smart garment according to the disclosed embodiment includes: receiving, at a user terminal, a design of a smart garment including a color, an image, text, and combinations thereof from a server; displaying the received design on the user terminal; and when the displayed design is selected, transmitting the selected design from the user terminal to the smart garment so that a design of the smart garment is changed to the selected design.", + "p0065": "Further, the method may further include: searching for a smart garment in the user terminal; displaying a found smart garment in the user terminal; and when the displayed smart garment is selected, pairing the smart garment with the user terminal in the user terminal.", + "p0066": "Further, the receiving of the design may include: when the user terminal and the smart garment are paired, receiving, at the user terminal, identification information transmitted from the smart garment; transmitting, at the user terminal, the received identification information to the server; and receiving, at the user terminal, a design of a smart garment transmitted from the server corresponding to the identification information.", + "p0067": "A smart garment according to the disclosed embodiments can change a design element, such as a color, an image, or text, so that a user can change a design of the smart garment to a desired design, regardless of time and place.", + "p0068": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0069": "FIG. 1 is a conceptual view of a system according to one illustrated embodiment.", + "p0070": "FIGS. 2 and 3 are diagrams illustrating a configuration of a smart garment according to one illustrated embodiment.", + "p0071": "FIG. 4 is a flowchart illustrating a method of changing a design of the smart garment of a system according to the disclosed embodiment.", + "p0072": "FIG. 5 is a diagram illustrating a user interface for changing a design of the smart garment according to the disclosed embodiment.", + "p0073": "FIG. 6 is a flowchart illustrating a method of changing a design of a smart garment of a system according to another disclosed embodiment.", + "p0074": "FIGS. 7 and 8 are diagrams illustrating a user interface for searching for and pairing with the smart garment displayed on a user terminal according to the disclosed embodiment.", + "p0075": "FIG. 9 is a diagram illustrating a user interface for changing a design of the smart garment according to another embodiment.", + "p0076": "FIG. 10 is a flowchart illustrating a method of changing a design of a smart garment of a system according to still another disclosed embodiment, and FIGS.", + "p0077": "FIGS. 11 to 16 are diagrams illustrating a user interface for changing a design of the smart garment according to still another disclosed embodiment.", + "p0078": "FIGS. 17 a to 20 e are diagrams illustrating cholesteric liquid crystal fibers among the color-changing fibers of the smart garment according to the disclosed embodiment.", + "p0079": "FIGS. 21 a , 21 b , 22 a and 22 b illustrate a connection structure of the color-changing fiber and the fabric of the smart garment according to the disclosed embodiment.", + "p0080": "FIGS. 23 a and 23 b are diagrams illustrating a structure of the fabric of the smart garment according to the disclosed embodiment.", + "p0081": "FIG. 24 is a diagram illustrating a connection relationship of color-changing fibers and the connection module for control of the color-changing fibers of the smart garment according to the disclosed embodiment.", + "p0082": "FIG. 25 is a diagram illustrating electronic ink fibers among the color-changing fibers of the smart garment according to the disclosed embodiment.", + "p0083": "FIG. 26 is a diagram illustrating an electrochromic fiber among color-changing fibers of the smart garment according to the disclosed embodiment.", + "p0084": "FIGS. 27 a , 27 b , 27 c , and 27 d are diagrams illustrating electroluminescent fibers among color-changing fibers of the smart garment according to the disclosed embodiment.", + "p0085": "DETAILED DESCRIPTION", + "p0086": "Hereinafter, disclosed embodiments will be described in detail with reference to the accompanying drawings.", + "p0087": "FIG. 1 is a conceptual view of a system 10 according to one illustrated embodiment, and FIGS. 2 and 3 are diagrams illustrating a configuration of a smart garment 400 according to one illustrated embodiment.", + "p0088": "Referring to FIG. 1 , the system 10 according to the illustrated embodiment includes the smart garment 400 , a provider terminal 100 configured to upload a design implementable on the smart garment 400 to a server 200 , the server 200 configured to store the design uploaded by the provider terminal 100 , and a user terminal 300 configured to download the design implementable on the smart garment 400 from the server 200 and transmit the design to the smart garment 400 .", + "p0089": "The design of the smart garment 400 refers to a color, an image, text, or a combination thereof, which is displayable on the smart garment 400 .", + "p0090": "A provider of the smart garment 400 , such as a seller or a business operator of the smart garment 400 , may upload the design implementable on the smart garment 400 to the server 200 using the provider terminal 100 . In addition to the above-described garment provider, a separate design provider or purchaser may freely upload a design of the smart garment 400 to the server 200 .", + "p0091": "The provider terminal 100 may include a computer capable of communicating with the server 200 or a mobile device such as a smart phone or a tablet personal computer . The devices are merely examples of the provider terminal 100 , and any device may be included in the range of the provider terminal 100 as long as the device is capable of communicating with the server 200 .", + "p0092": "The provider of the smart garment 400 uploads identification information of the smart garment 400 on which the uploaded design can be implemented, for example, a product number, a trademark, a size, a material, and the like, when the design is uploaded to the server 200 so that a design that can be implemented on the smart garment 400 possessed by the purchaser can be easily retrieved.", + "p0093": "The server 200 stores designs of the smart garment 400 uploaded from various provider terminals 100 to build a database. The server 200 may classify and store designs by providers who have uploaded the designs and classify and store designs uploaded by the same provider according to the type of smart garment 400 . A user may access the server 200 through the user terminal 300 and download the design from the server 200 after undergoing a predetermined authentication process.", + "p0094": "When the user inputs the identification information of the smart garment 400 , for example, the product number, the trademark, the size, the material, and the like, through the user terminal 300 after undergoing the authentication process, the user may download a design of the smart garment 400 associated with the input identification information from the server 200 to the user terminal 300 .", + "p0095": "The user terminal 300 may include a computer capable of communicating with the server 200 or a mobile device such as a smart phone or a tablet PC. The devices are merely examples of the user terminal 300 , and any device may be included in the range of the user terminal 300 as long as the device is capable of communicating with the server 200 . More specifically, the user terminal may include a communicator capable of communicating with the server or the smart garment, a display configured to display a user interface for changing a design of the smart garment, and a processor configured to generate a signal including a design of the smart garment and transmit the signal to the smart garment through the communicator when the design of the smart garment displayed on the display is selected.", + "p0096": "The user terminal 300 may serve as the provider terminal 100 when the design of the smart garment 400 is uploaded to the server 200 through the user terminal 300 , and even the provider terminal 100 may serve as the user terminal 300 when the provider terminal 100 downloads the design of the smart garment from the server 200 .", + "p0097": "The user terminal 300 transmits design information selected by the user from the design downloaded from the server 200 or a previously stored design to the smart garment 400 .", + "p0098": "The smart garment 400 includes a connection module 420 that may receive the design information transmitted from the user terminal 300 and change the design of the smart garment 400 based on the received information.", + "p0099": "As shown in FIG. 1 , when the connection module 420 of the smart garment 400 receives the signal including design information transmitted from the user terminal 300 , the smart garment 400 changes a color, displays or changes text, or displays or changes a pattern according to the received signal. That is, the user may implement various designs on one smart garment 400 .", + "p0100": "The connection module 420 of the smart garment 400 will be first described in detail with reference to FIGS. 2 and 3 , and a method of changing a design of the smart garment 400 and a user interface of the user terminal for changing a design of the smart garment 400 in the system according to one disclosed embodiment will be described in detail with reference to FIGS. 4 to 16 .", + "p0101": "Referring to FIG. 2 , the smart garment 400 includes the connection module 420 configured to receive a signal transmitted from the user terminal 300 . The connection module 420 may be provided at any position on the smart garment 400 . However, it is preferable for the connection module 420 to be provided at an edge of the garment which is easily connected to the user terminal 300 through a wired connection when considering the case of connecting to the user terminal 300 by wire.", + "p0102": "As shown in FIG. 2 , the connection module 420 may include a communicator 427 configured to perform wired/wireless communication with the user terminal 300 , a processor 423 configured to generate a control signal for changing a design of the smart garment 400 according to a signal received by the communicator 427 , a driver 421 configured to apply a voltage to the smart garment 400 according to the control signal generated by the processor 423 , a memory 425 configured to store design information included in the signal received by the communicator 427 or information related to a current design of the smart garment 400 , and a battery configured to supply power to the smart garment 400 .", + "p0103": "The communicator 427 of the connection module 420 is connected to the user terminal 300 through a communication scheme, such as a wireless local area network , Wi-Fi, Bluetooth, ZigBee, an ultra-wideband , infrared data association , Bluetooth low energy , near field communication , and the like.", + "p0104": "The driver 421 is connected to electrodes of color-changing fibers constituting the smart garment 400 , which will be described below, and applies a voltage to the electrodes according to the signal generated by the processor 423 so that the design of the smart garment 400 is changed. Only one driver 421 may be provided or a plurality of drivers 421 may be provided.", + "p0105": "The memory 425 may include not only a volatile memory, such as a static random access memory , a dynamic RAM , and the like, but also a flash memory, such as a flash memory, a read only memory , an erasable programmable read only memory , an electrically erasable programmable read only memory , and the like.", + "p0106": "As shown in FIG. 2 , the connection module 420 may include all or at least one of the communicator 427 , the processor 423 , the driver 421 , and the battery 429 .", + "p0107": "For example, the connection module 420 may include the communicator 427 and the driver 421 . In this case, the communicator 427 may receive the control signal generated by the processor of the user terminal 300 , and the driver 421 may change the design of the smart garment 400 by applying a voltage to the smart garment 400 according to the control signal received by the communicator 427 .", + "p0108": "Alternatively, the connection module 420 may include the communicator 427 , the processor 423 , and the driver 421 . In this case, when the communicator 427 receives the signal including the design information transmitted from the user terminal 300 , the processor 423 generates the control signal for controlling the driver 421 according to the signal received by the communicator 427 and outputs the control signal to the driver 421 . The driver 421 may change the design of the smart garment 400 by applying a voltage to the smart garment 400 according to the control signal output from the processor 423 of the connection module 420 .", + "p0109": "Alternatively, the connection module may include the communicator 427 , the processor 423 , the driver 421 , and the memory 425 . In this case, when the communicator 427 receives the signal including the design information transmitted from the user terminal 300 , the memory 425 stores the design information included in the signal received by the communicator 427 . Also, the memory 425 may store information related to a current design of the smart garment 400 . In addition, the processor 423 generates the control signal for controlling the driver 421 according to the signal received by the communicator 427 and outputs the control signal to the driver 421 . The driver 421 may change the design of the smart garment 400 by applying the voltage to the smart garment 400 according to the control signal output from the processor 423 .", + "p0110": "Alternatively, the connection module 420 may include the communicator 427 , the processor 423 , the driver 421 , the memory 425 , and the battery 429 . When light emitting diodes are used as the color-changing fibers constituting the smart garment 400 , the battery 429 may be needed to supply power to the light emitting didoes. In addition, when cholesteric liquid crystals, electronic ink, or electrochromic devices having bistability are used as fabrics constituting the smart garment 400 , the battery 429 may be omitted. Descriptions of the remaining configurations are the same as the above descriptions, and thus will be omitted.", + "p0111": "Alternatively, as shown in FIG. 3 , the connection module 420 of the smart garment 400 according to the disclosed embodiment may include only a connector 428 provided to be connected to the user terminal 300 by a wire. When the user terminal 300 and the smart garment 400 are connected by the connector 428 , the processor of the user terminal 300 may generate a signal for implementing a design selected by the user on the smart garment 400 , and the driver 421 of the user terminal 300 may apply a voltage for controlling the smart garment 400 through the connector 428 according to the signal generated by the processor 423 . When the signal for controlling the smart garment 400 is applied from the user terminal 300 through the connector 428 , the smart garment 400 may be changed to the design selected by the user.", + "p0112": "Meanwhile, FIG. 4 is a flowchart illustrating a method of changing a design of the smart garment 400 of a system according to the disclosed embodiment, and FIG. 5 is a diagram illustrating a user interface for changing a design of the smart garment 400 according to the disclosed embodiment.", + "p0113": "An application for changing a design of the smart garment 400 may be installed in the user terminal 300 . The application may provide a user interface related to pairing the smart garment 400 with the user terminal 300 , changing a design of the smart garment 400 , and the like through a display of the user terminal 300 .", + "p0114": "Referring to FIG. 4 , when the application of the user terminal 300 is executed, the user terminal 300 searches for a nearby smart garment 400 . The application may be executed by a command of a user, or may be automatically executed when the nearby smart garment 400 is sensed. An autorun function of the application may be changed through settings of the application.", + "p0115": "The user terminal 300 searches for the smart garment 400 and transmits a signal to the nearby smart garment 400 to be paired with the smart garment 400 . When the nearby smart garment 400 receives the signal transmitted from the user terminal 300 and transmits a signal in response to the received transmitted signal to the user terminal 300 , the user terminal 300 and the smart garment 400 are paired. As described above, the pairing of the smart garment 400 and the user terminal 300 may be performed through a communication scheme, such as a LAN, Wi-Fi, Bluetooth, ZigBee, a UWB, IrDA, BLE, NFC, and the like.", + "p0116": "FIGS. 4 and 5 illustrate a case in which one smart garment 400 is found. A case in which a plurality of nearby smart garments 400 are found will be described below, and searching and pairing of the smart garment 400 will be described below in detail. As shown in FIGS. 4 and 5 , when there is one nearby smart garment 400 , the user interface shown in FIG. 5 is displayed on the display of the user terminal 300 once the user terminal 300 and the smart garment 400 are paired.", + "p0117": "The user interface may include a garment display area 310 at an upper portion thereof where an image of the paired smart garment 400 is displayed, a color selection area 320 displayed below the garment display area 310 and provided to allow a color of the smart garment 400 displayed in the garment display area 310 to be selected, an image selection area 330 displayed below the color selection area 320 and provided to allow an image to be implemented on the smart garment 400 displayed in the garment display area 310 to be selected, and a text input area 340 provided below the image selection area 330 and provided to allow text to be displayed on the smart garment 400 displayed in the garment display area 310 to be input. Positions of the garment display area 310 , the color selection area 320 , the image selection area 330 , and the text input area 340 are not limited to the above-described positions, and may be variously set. For example, the user may change the positions as desired through application settings. The user may select a desired color or image or may input text through the user interface, and the user terminal 300 transmits a signal including information selected or input through the user interface to the smart garment 400 so that the design of the smart garment 400 can be changed. The connection module 420 of the smart garment 400 according to the present embodiment may include at least the connector 428 , as shown in FIG. 3 , or at least the communicator 427 and the driver 421 among the configuration shown in FIG. 2 .", + "p0118": "The image of the smart garment 400 displayed in the garment display area 310 may be a representative image set as a default in association with the paired smart garment 400 . For example, when the paired smart garment 400 is a short-sleeved t-shirt, a representative image that does not reflect a shape or the like of the paired short-sleeved t-shirt and is designed to allow the user to intuitively recognize that the smart garment is a short-sleeved t-shirt may be displayed in the garment display area 310 . According to another embodiment, an image similar to the actual smart garment 400 reflecting a current design of the smart garment 400 may be displayed in the garment display area 310 , which will be described below with reference to FIG. 6 .", + "p0119": "The color selection area 320 may be displayed in various colors in a matrix form, as shown in FIG. 5 . The user may change the color of the smart garment 400 by touching a desired color among the colors displayed in the color selection area 320 or by selecting the color through a separate inputter, such as a keyboard or a mouse. Once the color is selected, the changed color may be reflected in the image of the smart garment 400 displayed in the garment display area 310 . A manner in which colors are displayed in the color selection area 320 is not limited to the above-described example, and may be variously set. The user may change a display format of the colors through the application setting, and may set frequently used colors to be preferentially displayed.", + "p0120": "The image selection area 330 may display images including a pattern or design stored in advance in the user terminal 300 in association with the paired smart garment 400 , as shown in FIG. 5 . Like the image selection area 330 or the color selection area 320 , various images may be displayed in a matrix form. The user may change the image implemented on the smart garment 400 by touching a desired image among the images displayed in the image selection area 330 , or by selecting the image through a separate inputter, such as a keyboard or a mouse. Once the image is selected, the changed image may be reflected in the image of the smart garment 400 displayed in the garment display area 310 .", + "p0121": "As described above, in addition to the images stored in advance in the user terminal 300 in association with the paired smart garment 400 , images, such as photographs stored in the user terminal 300 , may also be displayed in the image selection area 330 . In addition, an object for driving a camera may also be displayed to display an image directly captured by the camera of the user terminal 300 on the smart garment 400 . That is, when the user touches the object or clicks the object through an inputter, the camera installed in the user terminal 300 is driven so that the user can acquire a desired image by camera capturing. When camera capturing is performed, the captured image may be displayed on the smart garment 400 displayed in the garment display area 310 . The user may select an image displayed in the image selection area 330 through application settings. That is, only the images stored in advance in the user terminal 300 in association with the paired smart garment 400 may be set to be displayed, or only images such as photos stored in the user terminal 300 may be displayed. In addition, only the object for driving the camera may be set to be displayed so that the image directly captured by the camera of the user terminal 300 can be displayed on the smart garment 400 .", + "p0122": "In the text input area 340 , a representation, such as <text input>, for allowing the user to know that a corresponding area is provided for text input may be displayed. Alternatively, the text input area 340 may be displayed as a blank space without a separate guide. When the user touches the text input area 340 or clicks it through a separate inputter, the user terminal 300 displays a keyboard for text input to enable the user to input desired text. A detailed description thereof will be given below. Text may be input through the keyboard or may be input via voice. The user may input text through the keyboard or input text via voice by application settings. An icon for guiding a keyboard input of text and an icon for guiding a voice input of text may be displayed in the text input area 340 so that the user can select a desired text input method by touching or clicking the icon for guiding the desired text input method.", + "p0123": "When text is input to the text input area 340 , the input text may be displayed on the smart garment 400 displayed in the garment display area 310 .", + "p0124": "FIG. 6 is a flowchart illustrating a method of changing a design of a smart garment 400 of a system according to another disclosed embodiment, FIGS. 7 and 8 are diagrams illustrating a user interface for searching for and pairing with the smart garment 400 displayed on a user terminal 300 according to the disclosed embodiment, and FIG. 9 is a diagram illustrating a user interface for changing a design of the smart garment 400 according to another embodiment.", + "p0125": "Referring to FIG. 6 , when an application of the user terminal 300 is executed, the user terminal 300 searches for a nearby smart garment 400 . The application may be executed by a command of a user, or may be automatically executed when the nearby smart garment 400 is sensed. An autorun function of the application may be changed through settings of the application.", + "p0126": "The user terminal 300 searches for the smart garment 400 and transmits a signal to the nearby smart garment 400 to be paired with the smart garment 400 . When the nearby smart garment 400 receives the signal transmitted from the user terminal 300 and transmits a signal in response to the received transmitted signal to the user terminal 300 , the user terminal 300 and the smart garment 400 are paired.", + "p0127": "As shown in FIGS. 7 and 8 , the user terminal 300 may provide the user interface for searching for the smart garment 400 and pairing with the found smart garment 400 . As shown in FIG. 7 , the user interface may display a garment display area 311 , a search button for searching for the nearby smart garment 400 below the garment display area 311 , and a pairing button 314 for pairing with the found smart garment 400 in a display of the user terminal 300 . Positions of the garment display area 311 , the search button 312 , and the pairing button 314 are not limited to the above-described positions, and may be variously set. For example, the user may change the positions as desired through application settings.", + "p0128": "When the user touches or clicks the search button 312 to search for the nearby smart garment 400 , as shown in FIG. 7 , an image of a magnifying glass may be displayed in the garment display area 311 so that the user can intuitively recognize that the nearby smart garment 400 is being searched for.", + "p0129": "When the nearby smart garment 400 is found, images of the found smart garments 400 may be displayed in a matrix form in the garment display area 311 , as shown in FIG. 8 . The user may select the smart garment 400 desired to be paired with among the smart garments 400 displayed in the garment display area 311 by touching or clicking the desired smart garment 400 , and may pair the selected smart garment 400 with the user terminal 300 by touching or clicking the pairing button 314 .", + "p0130": "As shown in FIG. 6 , when the smart garment 400 and the user terminal 300 are paired, the smart garment 400 extracts a current color thereof and a current image or text implemented on the smart garment 400 and transmits them to the user terminal 300 . The connection module 420 of the smart garment 400 according to the present embodiment may include at least the communicator 427 , the processor 423 , the driver 421 , and the memory 425 , and design information of the current smart garment 400 may be stored in the memory 425 .", + "p0131": "The user terminal 300 may display an image similar to the actual smart garment 400 reflecting the current design of the paired smart garment 400 or the like in the garment display area 310 on the basis of information transmitted from the smart garment 400 . For example, as shown in FIG. 9 , the user terminal 300 may display an image including a color of the paired smart garment 400 and a pattern or design implemented on the smart garment 400 , or, when text is implemented, an image reflecting the text in the garment display area 310 . A description of the other user interface shown in FIG. 9 is the same as that shown in FIG. 5 , and thus will be omitted.", + "p0132": "The user may select a desired color or image or input text through the user interface, and the user terminal 300 may transmit a signal including the information selected or input through the user interface to the smart garment 400 so that the design of the smart garment 400 can be changed.", + "p0133": "FIG. 10 is a flowchart illustrating a method of changing a design of a smart garment 400 of a system according to still another disclosed embodiment, and FIGS. 11 to 16 are diagrams illustrating a user interface for changing a design of the smart garment 400 according to still another disclosed embodiment.", + "p0134": "Referring to FIG. 10 , when an application of the user terminal 300 is executed, the user terminal 300 searches for a nearby smart garment 400 . The application may be executed by a command of a user, or may be automatically executed when the nearby smart garment 400 is sensed. An autorun function of the application may be changed through settings of the application.", + "p0135": "The user terminal 300 searches for the smart garment 400 and transmits a signal to the nearby smart garment 400 to be paired with the smart garment 400 . When the nearby smart garment 400 receives the signal transmitted from the user terminal 300 and transmits a signal in response to the received transmitted signal to the user terminal 300 , the user terminal 300 and the smart garment 400 are paired.", + "p0136": "A description of paring of the user terminal 300 and the smart garment 400 is the same as the description of FIGS. 6 to 8 , and thus will be omitted.", + "p0137": "When the connection module 420 of the smart garment 400 is paired with the user terminal 300 , the connection module 420 transmits identification information of the smart garment 400 to the user terminal 300 and the user terminal 300 transmits the identification information of the smart garment 400 transmitted from the smart garment 400 to the server 200 . The connection module 420 of the smart garment 400 according to the present embodiment may include at least the communicator 427 , the processor 423 , the driver 421 , and the memory 425 , and design information or identification information of the current smart garment 400 may be stored in the memory 425 .", + "p0138": "The server 200 searches for a design of the smart garment 400 represented by the identification information on the basis of the identification information of the smart garment 400 transmitted from the user terminal 300 and transmits found design information to the user terminal 300 . The user terminal 300 downloads the design of the paired smart garment 400 from the server 200 and displays the downloaded design in the color selection area 320 and the image selection area 330 of the user interface shown in FIG. 11 . Text implementable on the smart garment 400 may also be displayed as an image in the image selection area 330 .", + "p0139": "The user may select a desired color or image or input text through the user interface, and the user terminal 300 may transmit a signal including the information selected or input through the user interface to the smart garment 400 so that the design of the smart garment 400 can be changed.", + "p0140": "A method of changing a design of the paired smart garment 400 through the user interface provided by the user terminal 300 will be described in detail with reference to FIGS. 11 to 15 .", + "p0141": "As shown in FIGS. 11 and 12 , a user interface of the user terminal 300 is similar to the user interface shown in FIG. 5 . The user interface shown in FIGS. 11 and 12 is different from the user interface shown in FIG. 5 in that an interface for changing the smart garment 400 displayed in the garment display area 310 to another found smart garment 400 and an interface 315 provided for a user to recognize that the smart garment 400 displayed in the garment display area 310 is a paired garment when the smart garment 400 is paired with the user terminal 300 are further displayed in the garment display area 310 .", + "p0142": "When the user intends to change the smart garment 400 displayed in the garment display area 310 to another found smart garment 400 , as shown in FIG. 12 , the user may touch or click the interface for changing the smart garment 400 . Alternatively, the user may input a drag gesture or a flick gesture among touch gestures to the garment display area 310 to change the smart garment 400 displayed in the garment display area 310 to another smart garment 400 .", + "p0143": "When the smart garment 400 displayed in the garment display area 310 is changed, the color selection area 320 , the image selection area 330 , and the text input area 340 are changed to suit the changed smart garment 400 . For example, when the changed smart garment 400 does not support text input, a representation that conveys a meaning that text input is not possible, for example, <none>, is displayed in the text input area 340 , as shown in FIG. 12 .", + "p0144": "Also, as shown in FIG. 12 , when the changed smart garment 400 is a smart garment that is not paired with the user terminal 300 , the user terminal 300 does not display the interface indicating the pairing. In addition, when the smart garment 400 displayed in the garment display area 310 is changed to the paired smart garment 400 , the user terminal 300 displays an interface indicating that the displayed smart garment 400 is paired with the user terminal 300 .", + "p0145": "Meanwhile, when the user intends to implement an image on the paired smart garment 400 or change an existing image, the user may change the image of the smart garment 400 by touching or clicking the image, pattern, or design displayed in the image selection area 330 . As shown in FIG. 13 , an object for selecting photographs stored in the user terminal 300 may be displayed in the image selection area 330 in addition to the images transmitted from the server 200 .", + "p0146": "As shown in FIG. 13 , when an object allowing for connection to the images stored in the user terminal 300 is touched or clicked, the images stored in the user terminal 300 may be displayed in a matrix form below the garment display area. In addition, an area 316 in which an image can be displayed is displayed in a rectangular shape on the smart garment 400 displayed in the garment display area 310 so that the user can recognize where the image selected by the user is displayed on the smart garment 400 . The user may change a shape, size, or position of the area displayed on the smart garment 400 in the garment display area 310 .", + "p0147": "As shown in FIG. 13 , when an image representing a flower is selected through a touch or click, the selected image is displayed on the smart garment 400 displayed in the garment display area 310 . In addition, the user terminal 300 transmits a signal including information for implementing the selected image on the smart garment 400 to the smart garment 400 so that the selected image is displayed on the smart garment 400 . As shown in FIG. 13 , when the image is selected, the user terminal 300 may directly forward the signal for implementing the image to the smart garment 400 , or may display a message requesting confirmation of the application of the selected image and forward the signal for implementing the image to the smart garment 400 when a confirmation command is input.", + "p0148": "In addition, as shown in FIG. 14 , an object for driving the camera may also be displayed in the image selection area 330 so that an image directly captured by the camera of the user terminal 300 can be displayed on the smart garment 400 .", + "p0149": "As shown in FIG. 14 , when the object for driving the camera of the user terminal 300 is touched or clicked, a camera installed in the user terminal 300 is driven so that the user can acquire a desired image through camera capturing. Once camera capturing is performed, a captured image may be displayed on the smart garment 400 displayed in the garment display area 310 . The user may adjust a position or size of the captured image by changing a shape, size, or position of the image display area displayed on the smart garment 400 . When the above adjustment is complete, the user terminal 300 transmits a signal including information for implementing the captured image on the smart garment 400 to the smart garment 400 so that the captured image is displayed on the smart garment 400 . As shown in FIG. 16 , when an image is captured and adjustment of a size or position of the image is complete, the user terminal 300 \u2032 may directly forward a signal for implementing the captured image on the smart garment 400 , or may display a message requesting confirmation of application of the captured image and then forward the signal for implementing the image to the smart garment 400 when a confirmation command is received.", + "p0150": "Meanwhile, the user may input text through the text input area 340 when the user intends to display text on the paired smart garment 400 or change existing text.", + "p0151": "As shown in FIG. 15 , in the text input area 340 , a representation, such as <text input>, may be displayed that allows the user to know that a corresponding area is provided for text input. Alternatively, the text input area 340 may be displayed as a blank space without a separate guide.", + "p0152": "When the user touches the text input area 340 or clicks the text input area 340 through a separate inputter, the user terminal 300 displays an area provided for selecting a color of text and a keyboard for inputting text, as shown in FIG. 15 , so that the user can input text of a desired color. In addition, an area 341 in which text can be displayed is displayed in a rectangular shape on the smart garment 400 displayed in the garment display area 310 so that the user can recognize where the image selected by the user is displayed on the smart garment 400 . The user may change a shape, size, or position of the text display area displayed on the smart garment 400 in the garment display area 310 .", + "p0153": "In addition, although not illustrated in the drawings, a user interface for selecting a text font may be further displayed to allow the user to select a desired font.", + "p0154": "When text is input and a color and font of the input text are selected, the input text is displayed on the smart garment 400 displayed in the garment display area 310 . Also, the user terminal 300 transmits a signal including information for implementing the input text on the smart garment 400 to the smart garment 400 so that the input text can be displayed on the smart garment 400 . As shown in FIG. 15 , when text is input, the user terminal 300 may directly forward a signal for displaying the text to the smart garment 400 , or may display a message requesting confirmation of application of the input text and then forward the signal for displaying the text to the smart garment 400 when a confirmation command is input.", + "p0155": "Further, the text may be input through a keyboard or input via voice. The user may input text through the keyboard or via voice by application settings. An icon for guiding keyboard input of text and an icon for guiding voice input of text may be displayed in the text input area 340 so that the user can select a desired text input method by touching or clicking the icon for guiding the desired text input method. When the icon for guiding keyboard input is selected, the user interface shown in FIG. 15 may be displayed.", + "p0156": "When text is input to the text input area 341 , the input text may be displayed on the smart garment 400 displayed in the garment display area 310 .", + "p0157": "As described above, the design of the smart garment 400 may be changed through the user interface provided by the user terminal 300 , or the user terminal 300 may sense information of the surrounding environment and automatically change the design of the smart garment 400 , which will be described with reference to FIG. 16 .", + "p0158": "As shown in FIG. 16 , the user terminal 300 may recognize a position of the user terminal 300 or detect a surrounding temperature or weather using a sensor mounted on the user terminal 300 or by utilizing Internet information or the like. The position, temperature, and weather are examples of environment information that the user terminal 300 can detect, but the present invention is not limited thereto.", + "p0159": "For example, when a current position recognized by the user terminal 300 is a baseball field, the user terminal 300 may transmit a signal to the smart garment 400 such that a symbol of a team playing a game at the baseball field can be implemented on the smart garment 400 . The smart garment 400 may display the symbol of the team according to the signal transmitted from the user terminal 300 .", + "p0160": "When a current temperature sensed by the user terminal 300 is more than 30\u00b0 C., the user terminal 300 may transmit a signal for changing a color of the current smart garment 400 to white to the smart garment 400 . The smart garment 400 may change the color thereof to white according to the signal transmitted from the user terminal 300 . When the color of the current smart garment 400 is already white, the smart garment 400 may not change the color.", + "p0161": "In addition, when a current weather detected by the user terminal 300 is rainy weather, the user terminal 300 may transmit a signal for changing a pattern, a design, or an image currently displayed on the smart garment 400 to a raindrop image, design, or pattern to the smart garment 400 . The smart garment 400 may change the current pattern, design, or image thereof to the raindrop pattern, design, or image according to the signal transmitted from the user terminal 300 .", + "p0162": "An application may provide a surrounding environment recognition mode that provides the above-described functions, and the user may activate the surrounding environment recognition mode to automatically cause the user terminal 300 to change the design of the smart garment 400 . Alternatively, the surrounding environment recognition mode may be usually turned off and optionally activated in a desired situation.", + "p0163": "Meanwhile, the smart garment 400 according to the disclosed embodiment includes a fabric including color-changing fibers and ordinary fibers, which are materials for changing an image, text, or color according to a signal transmitted from the user terminal 300 , and the above-described connection module 420 . Hereinafter, the color-changing fibers constituting the smart garment 400 will be described in detail.", + "p0164": "FIGS. 17 to 20 are diagrams illustrating cholesteric liquid crystal fibers among the color-changing fibers of the smart garment 400 according to the disclosed embodiment.", + "p0165": "As shown in FIG. 17 A , cholesteric liquid crystal fiber 419 may have a circular or polygonal cross-section. As shown in FIG. 17 A , the cholesteric liquid crystal fiber has a cholesteric liquid crystal 415 provided at the center of the fiber, a first electrode 412 a formed on a part of an outer surface of the cholesteric liquid crystal, a second electrode 412 b formed at an opposite surface of the first electrode, an insulator 413 provided between the first electrode and the second electrode, and a protective layer 411 covering the first electrode, the second electrode, and the insulator.", + "p0166": "Cholesteric liquid crystals have bistability such that the cholesteric liquid crystals may be present in two stable states, such as a planar state in which light is reflected even when no voltage is applied to the cholesteric liquid crystals and a focal conic state in which light is scattered. The cholesteric liquid crystals may be converted into a homeotropic state in which light can be transmitted when a high voltage is applied thereto. Thus, when the design of the smart garment 400 is changed by the user terminal 300 , the changed design may be maintained without the application of voltage.", + "p0167": "An inner electrode and an outer electrode may be transparent electrodes made of a transparent conductive material, and examples of the transparent conductive material may include indium tin oxide , indium zinc oxide , aluminum-doped zinc oxide , a silver nano-wire , and the like.", + "p0168": "The protective layer covering the first electrode, the second electrode, and the insulator may be formed by coating ordinary textile fibers or fibers in a mesh form. By using ordinary textile fibers as the protective layer, a texture of ordinary textile fibers may be imparted to the cholesteric liquid crystal fiber. The first electrode and the second electrode may be connected to the driver 421 of the connection module 420 , and the driver 421 may change the design of the smart garment 400 by applying a voltage to the first electrode and the second electrode according to a control signal of the processor 423 .", + "p0169": "As shown in FIG. 17 B , the cholesteric liquid crystal fiber may include a partition wall 416 for maintaining a shape of the fiber in the cholesteric liquid crystals. The partition wall may be formed by a polymer structure. In addition, as shown in FIG. 17 C , the cholesteric liquid crystal fiber may be provided to have a polygonal cross-section, such as a quadrangular cross-section.", + "p0170": "As shown in FIG. 18 , the above-described cholesteric liquid crystal fiber may be combined with an ordinary textile fiber 490 to form a color-changing fiber constituting the smart garment 400 . As shown in FIG. 18 , the cholesteric liquid crystal fiber and the ordinary textile fiber may be formed to have a twisted structure. A fabric of the smart garment 400 according to the disclosed embodiment may be formed by weaving color-changing fibers having a twisted form of cholesteric liquid crystal fibers and textile fibers.", + "p0171": "In the case of the fabric woven with the color-changing fibers of the structure shown in FIG. 18 , the above-described first electrode and second electrode may not be formed on the cholesteric liquid crystal fiber, but the first electrode and the second electrode may be formed on one surface of the woven fabric and an opposite surface thereof.", + "p0172": "In another embodiment, as shown in FIG. 19 , the fabric of the smart garment 400 may be formed by weaving the cholesteric liquid crystal fibers 430 and the ordinary textile fibers 422 in the form of a net. According to the present embodiment, unlike the embodiment shown in FIG. 18 , the fabric is formed by weaving the cholesteric liquid crystal fibers and the textile fibers in the form of a net without weaving the fabric using color-changing fibers having a twisted structure of cholesteric liquid crystal fibers and textile fibers.", + "p0173": "FIG. 20 A illustrates a structure of pixels P of a cholesteric liquid crystal fiber according to the disclosed embodiment. Referring to FIG. 20 A , the cholesteric liquid crystal fiber may have a structure in which red pixels reflecting one color of light, for example, red light, are arranged in a single layer. In this case, since one cholesteric liquid crystal fiber reflects one color, cholesteric liquid crystal fibers arranged in adjacent columns or in adjacent rows in the form shown in FIG. 19 may be provided to reflect different colors.", + "p0174": "Alternatively, as shown in FIG. 20 B , one pixel P constituting the cholesteric liquid crystal fiber may have a structure in which a red sub-cell 431 reflecting a wavelength band corresponding to red light, a green sub-cell 432 reflecting a wavelength band corresponding to green light, and a blue sub-cell 433 reflecting a wavelength band corresponding to blue light are stacked atop one another.", + "p0175": "As shown in FIG. 20 B , one pixel may have a stacked structure, or, as shown in FIGS. 20 C and 20 D , one pixel may have a structure in which a red sub-cell reflecting a wavelength band corresponding to red light, a green sub-cell reflecting a wavelength band corresponding to green light, and a blue sub-cell reflecting a wavelength band corresponding to blue light are arranged in a single layer.", + "p0176": "As shown in FIGS. 20 C and 20 D , a grid 434 that separates each of sub-cells constituting one pixel may be implemented by an ordinary textile fiber material so that cholesteric liquid crystal fibers may have a texture of textile fibers. The grid may be formed to have a height higher than the cholesteric liquid crystals, as shown in FIG. 20 C , or may be formed at the same height as the cholesteric liquid crystals, as shown in FIG. 20 D .", + "p0177": "As shown in FIG. 20 E , even when one pixel has a stacked structure, a grid separating each sub-cell may be implemented by an ordinary textile fiber material so that cholesteric liquid crystal fibers may have a texture of textile fibers.", + "p0178": "Meanwhile, when the smart garment 400 , for example, a T-shirt, is made, it is necessary to connect fabrics of a sleeve portion and a body portion. This connection may be made at a cholesteric liquid crystal fiber level or at a fabric level formed by cholesteric liquid crystal fibers. FIGS. 21 and 22 illustrate a connection structure of the color-changing fiber and the fabric of the smart garment 400 according to the disclosed embodiment.", + "p0179": "Referring to FIG. 21 A , since the cholesteric liquid crystal fiber includes the first electrode and the second electrode to which a voltage is applied, as described above, when cholesteric liquid crystal fibers 419 are connected to each other, electrodes of the same polarity should be electrically connected. That is, first electrodes of the cholesteric liquid crystal fibers to be connected should be electrically connected to each other, and second electrodes should be electrically connected to each other. As shown in FIG. 21 B , a connection portion 440 of the cholesteric liquid crystal fiber may include a first connection electrode 440 a which electrically connects the first electrodes 412 a of the cholesteric liquid crystal fibers, a second connection electrode 440 b which electrically connects the second electrodes 412 b of the cholesteric liquid crystal fibers, and an insulator provided between the first connection electrode and the second connection electrode. Additionally, an outer protective layer which may protect the connection portion may further included. A cross-section of the connection portion may be formed to be circular when a cross-section of the cholesteric liquid crystal fiber is circular, and the cross-section of the connection portion may be formed to be polygonal corresponding to the cross-section of the cholesteric liquid crystal fiber when the cross-section of the cholesteric liquid crystal fiber is polygonal.", + "p0180": "In the case of the fabric woven with the color-changing fibers having a twisted structure shown in FIG. 18 , the first electrode and the second electrode are formed in the woven fabric, as described above. In this case, as shown in FIG. 22 A , first electrodes of fabrics 450 to be connected should be electrically connected to each other, and second electrodes should be electrically connected to each other. As shown in FIG. 22 A , a connection portion 455 of the fabric may include a first connection electrode 455 a which connects the first electrodes 450 a formed on top surfaces of the fabrics to be connected and a second connection electrode 455 b which connects the second electrodes 450 b formed on bottom surfaces of the fabrics to be connected. FIG. 22 A illustrates a cross-section of the fabrics to be connected and the connection portion which connects the fabrics, and FIG. 22 B illustrates the top surface. In FIG. 22 B , outer electrodes illustrated by solid lines are the first connection electrodes and outer electrodes illustrated by dotted lines are the second connection electrodes.", + "p0181": "As described above, the fabric of the smart garment 400 may be formed using the color-changing fibers having the twisted structure shown in FIG. 18 , or may be formed by weaving cholesteric liquid crystal fibers and ordinary textile fibers in the form of a net as shown in FIG. 19 . As another method, as shown in FIGS. 23 A and 23 B , a flexible cholesteric liquid crystal in the form of a film may be used. FIG. 23 A is a diagram illustrating a structure of the fabric of the smart garment 400 according to the disclosed embodiment.", + "p0182": "As shown in FIG. 23 A , the flexible cholesteric liquid crystal 460 may be formed into a net shape by forming holes in the form of a matrix in the flexible cholesteric liquid crystal and inserting an ordinary textile fiber 455 into a net eye, i.e., the hole, such that the fabric of the smart garment 400 is formed. Alternatively, as shown in FIG. 23 B , textile fibers may be formed into a net shape by forming holes in the form of a matrix in the ordinary textile fibers and combining the flexible cholesteric liquid crystal 460 into a net eye, i.e., the hole, such that the fabric of the smart garment 400 is formed.", + "p0183": "The cholesteric liquid crystal illustrated in FIGS. 23 A and 23 B may include pixels of a stacked structure or pixels of a single layer structure, as described above. The cholesteric liquid crystals of FIG. 23 A may be individually controlled for each line, and the cholesteric liquid crystals of FIG. 23 B may also be individually controlled for each cholesteric liquid crystal forming the net eye.", + "p0184": "FIG. 24 is a diagram illustrating a connection relationship of color-changing fibers and the connection module 420 for control of the color-changing fibers of the smart garment 400 according to the disclosed embodiment. As shown in FIG. 24 , one of a first electrode 430 a - 1 and a second electrode 430 a - 2 of a cholesteric liquid crystal fiber 430 a as a first material and a cholesteric liquid crystal fiber 430 b as a second material that constitute a fabric may perform a role of a signal electrode, and the other may perform a role of a ground electrode. In the same manner, one of a third electrode 430 b - 1 and a fourth electrode 430 b - 2 may perform a role of a signal electrode, and the other may perform a role of a ground electrode. Since both the first material and the second material are cholesteric liquid crystal fibers, which are color-changing fibers, and the first electrode, the second electrode, the third electrode, and the fourth electrode perform the same function, the first electrode and the second electrode will be described as an example. For example, when the first electrode is the signal electrode and the second electrode is the ground electrode, the first electrodes of the cholesteric liquid crystal fibers are electrically connected to a signal terminal of the driver 421 of the connection module 420 , and the second electrodes are electrically connected to a ground terminal of the driver 421 of the connection module 420 . All the cholesteric liquid crystal fibers may be connected to one connection module 420 , or the cholesteric liquid crystal fibers may be controlled using a plurality of connection modules 420 .", + "p0185": "The cholesteric liquid crystal fiber has been described above as an example of the color-changing fiber for changing a design of the smart garment 400 . Hereinafter, other embodiments of the color-changing fiber will be described.", + "p0186": "FIG. 25 is a diagram illustrating electronic ink fibers among the color-changing fibers of the smart garment 400 according to the disclosed embodiment.", + "p0187": "As shown in FIG. 25 , the electronic ink fabric may include a plurality of electronic ink capsules 470 containing electronic ink, a first electrode 470 a formed on one surface of the electronic ink capsule, a second electrode 470 b formed on an opposite surface of the first electrode, and an insulator provided between the first electrode and the second electrode. In addition, although not illustrated in the drawings, a protective layer covering the first electrode, the second electrode, and the insulator may be further included.", + "p0188": "The electronic ink capsules may contain at least one of a red electronic ink reflecting red light, a green electronic ink reflecting green light, a blue electronic ink reflecting blue light, a black electronic ink that absorbs light, and a white electronic ink that reflects light. For example, the red electronic ink, the black electronic ink, and the white electronic ink may be contained in one capsule, the green electronic ink, the black electronic ink, and the white electronic ink may be contained in one capsule, and the blue electronic ink, the black electronic ink, and the white electronic ink may be contained in one capsule.", + "p0189": "An inner electrode and an outer electrode may be transparent electrodes made of a transparent conductive material, and examples of the transparent conductive material may include ITO, IZO, aluminum-doped ZAO, AgNW, and the like.", + "p0190": "The protective layer covering the first electrode, the second electrode, and the insulator may be formed by coating ordinary textile fibers or fibers in a mesh form. By using ordinary textile fibers as the protective layer, a texture of the ordinary textile fibers may be imparted to the electronic ink fiber. The first electrode and the second electrode may be connected to the driver 421 of the connection module 420 , and the driver 421 may change the design of the smart garment 400 by applying a voltage to the first electrode and the second electrode according to the control signal of the processor 423 .", + "p0191": "The fabric of the smart garment 400 according to the present embodiment may be formed by color-changing fibers having a twisted structure of the electronic ink fibers and the ordinary textile fibers, as in the embodiment using the cholesteric liquid crystal fibers, or may be formed by weaving the electronic ink fibers and the ordinary textile fibers in the form of a net. In the case of the fabric woven with the color-changing fibers having the twisted structure of the electronic ink fibers and the ordinary textile fibers, the above-described first electrode and second electrode may not be formed on the electronic ink fiber, but the first electrode and the second electrode may be formed on one surface of the woven fabric and an opposite surface.", + "p0192": "Since the electronic ink also has bistability like the above-described cholesteric liquid crystal, power required for changing a design of the smart garment 400 may be minimized.", + "p0193": "FIG. 26 is a diagram illustrating an electrochromic fiber among color-changing fibers of the smart garment 400 according to the disclosed embodiment.", + "p0194": "As shown in FIG. 26 , the electrochromic fiber 480 may include a first electrode 482 , a counter electrode 489 provided outside the first electrode, an electrolyte 487 provided outside the counter electrode, a working electrode 485 provided outside the electrolyte, a second electrode 483 provided outside the working electrode, and a protective layer 481 provided to cover the second electrode. In addition, although not illustrated in the drawings, a partition wall for maintaining a shape of the fiber may be provided inside the electrochromic fiber. In the present embodiment, since the first electrode will be referred to as an inner electrode since it is provided inside the fiber, and the second electrode will be referred to as an outer electrode.", + "p0195": "The inner electrode and the outer electrode may be transparent electrodes made of a transparent conductive material to which a voltage is applied from the driver 421 of the connection module 420 . Examples of the transparent conductive material may include ITO, IZO, aluminum-doped ZAO, AgNW, and the like.", + "p0196": "The electrolyte may be a solid electrolyte or a liquid electrolyte, which is ionized and supplies a charge or ion to the inner electrode and the outer electrode when a voltage is applied thereto.", + "p0197": "At least one of the working electrode and the counter electrode includes a color-changing material. For example, the working electrode may include a material that changes color as it is reduced, and the counter electrode may include a material that changes color as it is oxidized. Both the working electrode and the counter electrode may include the above-described color-changing material, or only one of the working and counter electrodes may include the above-described color-changing material while the other may include an ion-receiving electrode.", + "p0198": "Typical reducing color-changing materials include WO3 and MoO3 TiO2, and oxidative color-changing materials include Ir x, Ni 2, and Rh2O3. The electrochromic fiber may realize different red colors, green colors, and blue colors and combinations thereof in the color-changing material, and may be changed to a transparent state. The protective layer may be formed by coating ordinary textile fibers or fibers in a mesh form. By using ordinary textile fibers as the protective layer, a texture of the ordinary textile fiber may be imparted to the electrochromic fiber. The inner electrode and the outer electrode may be connected to the driver 421 of the connection module 420 , and the driver 421 may change the design of the smart garment 400 by applying a voltage to the first electrode and the second electrode according to a control signal of the processor 423 .", + "p0199": "A fabric of the smart garment 400 according to the present embodiment may be formed by mixing electrochromic fibers and ordinary textile fibers, as in the embodiment using the cholesteric liquid crystal fabric. Since the electrochromic material also has bistability like the above-described cholesteric liquid crystal, power required for changing a design of the smart garment 400 may be minimized.", + "p0200": "FIGS. 27 A, 27 B, 27 C, 27 D, and 27 E are diagrams illustrating electroluminescent fibers among color-changing fibers of the smart garment 400 according to the disclosed embodiment.", + "p0201": "The above-described cholesteric liquid crystal fibers, electronic ink fibers, and electrochromic fibers are reflective fibers, and electroluminescent fibers 490 shown in FIG. 27 A are self-luminescent fibers. Thus, when a fabric of the smart garment 400 is formed using the electroluminescent fibers, the connection module 420 of the smart garment 400 includes the battery 429 .", + "p0202": "As shown in FIG. 27 A , the electroluminescent fibers may include a first electrode 497 , a luminescent material 495 provided outside the first electrode, a second electrode 493 provided outside the luminescent material, and a protective layer 491 covering the second electrode 493 . Each of the electroluminescent fibers may use a luminescent material that emits red, green, or blue light, and fibers using these different luminescent materials may be woven together with ordinary textile fibers to form the fabric of the smart garment 400 .", + "p0203": "In the present embodiment, the first electrode will be referred to as an inner electrode since it is provided inside the fiber, and the second electrode will be referred to as an outer electrode.", + "p0204": "In another embodiment, as shown in FIG. 27 B , a material that emits white light is used as the luminescent material, and color filters 492 a , 492 b , and 492 c , which transmit red light, green light, and blue light, respectively, may be attached to outer surfaces of the outer electrodes to form the electroluminescent fiber. At this time, as shown in FIG. 27 B , the outer electrodes are spaced apart from each other, and the aforementioned color filters are attached respectively to the outer electrodes provided at a distance from each other. In this case, each of the outer electrodes may be individually controlled by the driver 421 of the connection module 420 . A partition wall 496 for maintaining a shape of the fiber may be provided inside the luminescent material.", + "p0205": "In still another embodiment, as shown in FIG. 27 C , materials 495 a , 495 b , and 495 c that emit red light, green light, and blue light, respectively, are used as the luminescent material, and an outer electrode is provided outside each of the materials to form the electroluminescent fiber. At this time, as shown in FIG. 27 C , the outer electrodes may be spaced apart from each other, and may be individually controlled by the driver 421 of the connection module 420 .", + "p0206": "In yet another embodiment, as shown in FIG. 27 D , a core of the electroluminescent fiber is formed by an organic textile fiber 494 , and three inner electrodes spaced apart from each other are formed outside the textile fiber. In addition, materials 495 a , 495 b , and 495 c , which emit red light, green light, and blue light, respectively, may be provided as luminescent materials formed outside of the inner electrodes, and outer electrodes may be provided respectively outside of the materials to form the electroluminescent fiber. In the present embodiment, inner and outer electrodes that apply voltages to different luminescent materials may be individually controlled by the driver 421 of the connection module 420 .", + "p0207": "The inner electrode and the outer electrode are transparent electrodes made of a transparent conductive material. Examples of the transparent conductive material may include ITO, IZO, aluminum-doped ZAO, AgNW, and the like. The inner electrode and the outer electrode may be formed in a planar shape or in a linear shape.", + "p0208": "A protective layer may be formed by coating ordinary textile fibers or fibers in a mesh form. By using ordinary textile fibers as the protective layer, a texture of the ordinary textile fibers may be imparted to the electroluminescent fiber. As described above, the inner electrode and the outer electrode may be connected to the driver 421 of the connection module 420 , and the driver 421 may change the design of the smart garment 400 by applying a voltage to the inner electrode and the outer electrode according to the control signal of the processor 423 .", + "p0209": "The fabric of the smart garment 400 according to the present embodiment may be formed by mixing the electrochromic fibers and the ordinary textile fibers, as in the embodiment using the cholesteric liquid crystal fibers.", + "features": { + "F1": "A smart garment comprising a first material with a first electrode and a second electrode.", + "F2": "A second material in the smart garment comprising a third electrode and a fourth electrode.", + "F3": "A connection module in the smart garment comprising a first ground terminal configured to ground one of the first electrode and the second electrode in response to a signal transmitted by a user terminal.", + "F4": "The connection module comprising a first signal terminal applying a voltage to another one of the first electrode and the second electrode.", + "F5": "The connection module comprising a second ground terminal configured to ground one of the third electrode and the fourth electrode.", + "F6": "The connection module comprising a second signal terminal applying a voltage to another one of the third electrode and the fourth electrode.", + "F7": "The first material and the second material comprise cholesteric liquid crystal fibers including a cholesteric liquid crystal.", + "F8": "The cholesteric liquid crystal comprises a pixel including a red sub-cell configured to reflect red light, a green sub-cell configured to reflect green light, and a blue sub-cell configured to reflect blue light.", + "F9": "A grid is provided to separate the red sub-cell, the green sub-cell, and the blue sub-cell, and is formed from textile fibers.", + "F10": "The pixel is provided so that the red sub-cell, the green sub-cell, and the blue sub-cell are either formed to be coplanar or to have a stacked structure.", + "F11": "Each cholesteric liquid crystal fiber further comprises a first cholesteric liquid crystal fiber electrode provided at one side of the cholesteric liquid crystal and a second cholesteric liquid crystal fiber electrode provided at a side opposite the first.", + "F12": "An insulator is provided between the first and second cholesteric liquid crystal fiber electrodes.", + "F13": "A protective layer is provided to cover the first cholesteric liquid crystal fiber electrode, the second cholesteric liquid crystal fiber electrode, and the insulator.", + "F14": "A connection portion is provided to connect the cholesteric liquid crystal fibers so as to extend them.", + "F15": "The connection portion comprises a first connection electrode provided to connect the first cholesteric liquid crystal fiber electrodes of the cholesteric liquid crystal fibers.", + "F16": "A second connection electrode is provided to connect the second cholesteric liquid crystal fiber electrodes of the cholesteric liquid crystal fibers.", + "F17": "An insulator is provided between the first and second connection electrodes.", + "F18": "The cholesteric liquid crystal comprises a partition wall provided to maintain a shape of the cholesteric liquid crystal fibers." + } + } + }, + { + "FAN": "61277994", + "CPC": [ + "G06F-003/06/44", + "G06F-016/00", + "G06F-016/10", + "G06F-016/113", + "G06F-016/14", + "G06F-016/1744", + "G06F-016/185", + "G06F-016/215", + "G06F-016/2453", + "G06F-016/90335", + "G06F-017/30", + "G06F-017/30/067", + "G06F-017/30/073", + "G06F-017/30/1", + "G06F-017/30/153", + "G06F-017/30/221", + "G06F-017/30/442" + ], + "Content": { + "title": "Archival data storage system", + "pa01": "A cost-effective, durable and scalable archival data storage system is provided herein that allow customers to store, retrieve and delete archival data objects, among other operations. For data storage, in an embodiment, the system stores data in a transient data store and provides a data object identifier may be used by subsequent requests. For data retrieval, in an embodiment, the system creates a job corresponding to the data retrieval and provides a job identifier associated with the created job. Once the job is executed, data retrieved is provided in a transient data store to enable customer download. In various embodiments, jobs associated with storage, retrieval and deletion are scheduled and executed using various optimization techniques such as load balancing, batch processed and partitioning. Data is redundantly encoded and stored in self-describing storage entities increasing reliability while reducing storage costs. Data integrity is ensured by integrity checks along data paths.", + "c-en-0001": "A computer-implemented method for data integrity verification, comprising: under control of one or more computer systems configured with executable instructions, receiving , from a remote computing device, one or more parts of a data payload; for each part of at least a subset of the one or more parts, generating a sub-part; calculating a digest for each generated sub-part based at least in part on the generated sub-part; generating a hierarchical data structure and storing the calculated digests as nodes of the data structure; generating at least a root node of the data structure based at least in part on the calculated digests by concatenating each calculated digest and running a hash function on the concatenated digest; determining a top-level digest of the data structure associated with the root node of the data structure by traversing the data structure to the root node; storing the data payload in archival data storage; validating , based at least in part on the determined top-level digest and a received top-level digest, that the stored data payload matches a received data payload corresponding to the received one or more parts; providing a data object identifier including the top-level digest to a user of the archival data storage; receiving a request for the stored data payload from the user including the data object identifier; verifying that the stored data payload has not changed; and providing the stored data payload to the user.", + "c-en-0002": "The computer-implemented method of claim 1, wherein the sub-part is generated to be a first predefined size, and wherein the received one or more parts are a second size that is an integer multiple of the first predefined size.", + "c-en-0003": "The computer-implemented method of claim 2, wherein the integer is based at least in part on a degree of branches of the data structure.", + "c-en-0004": "The computer-implemented method of any one of the preceding claims, further comprising: storing the data payload of a first storage device to a second storage device; and verifying, based at least in part on the top-level digest, that the data payload stored in the second storage device matches the data payload of the first storage device.", + "c-en-0005": "The computer-implemented method of any one of the preceding claims, further comprising: repartitioning the data payload based at least in part on a second generated sub-part, the second generated sub-part being different from the generated sub-part; and verifying, based at least in part on the top-level digest, that the repartitioned data payload matches a stored data payload, the stored data payload based at least in part on the received data payload.", + "c-en-0006": "The computer-implemented method of any one of the preceding claims, further comprising: verifying, based at least in part on the top-level digest, that a repartitioned data payload matches the received data payload, the repartitioned data payload based at least in part on a second sub-part of the one or more parts.", + "c-en-0007": "The computer-implemented method of claim 6, further comprising providing an instruction to a remote computing device, the instruction configured to enable the remote computing device to generate the data structure and provide the identified top-level digest.", + "c-en-0008": "The computer-implemented method of claim 6 or claim 7, wherein the one or more parts of the data payload are received as part of an electronic request, from a remote computing device, to store the data payload.", + "c-en-0009": "The computer-implemented method of claim 8, further comprising providing a function call for a remote device to make the electronic request, and wherein the electronic request includes at least the provided function call.", + "c-en-0010": "The computer-implemented method of any one of claims 6 to 9, further comprising receiving an indication of a size for each of the one or more parts of the data payload.", + "c-en-0011": "A system for verifying data integrity, comprising: at least one memory that stores computer-executable instructions; and at least one processor configured to access the at least one memory, wherein the at least one processor is configured to execute the computer-executable instructions to perform the method of any one of the preceding claims.", + "c-en-0012": "One or more computer-readable media storing computer-executable instructions for verifying data integrity that, when executed by one or more processors, configure the one or more processors to perform the method of any one of claims 1 to 10.", + "p0001": "BACKGROUND", + "p0002": "With increasing digitalization of information, the demand for durable and reliable archival data storage services is also increasing. Archival data may include archive records, backup files, media files and the like maintained by governments, businesses, libraries and the like. The archival storage of data has presented some challenges. For example, the potentially massive amount of data to be stored can cause costs to be prohibitive using many conventional technologies. Also, it is often desired that the durability and reliability of storage for archival data be relatively high, which further increases the amount of resources needed to store data, thereby increasing the expense. Conventional technologies such as magnetic tapes have traditionally been used in data backup systems because of the low cost. However, tape-based and other storage systems often fail to fully exploit advances in storage technology, such as data compression, error correction and the like, that enhance the security, reliability and scalability of data storage systems.", + "p0003": "WO 2008/083914 discloses a method, system, and computer program product for simultaneous multi-channel upload of a file to one or more servers while ensuring data integrity.", + "p0004": "US 2005/235154 A1 deals with techniques for storing, retrieving and validating data.", + "p0005": "SUMMARY", + "p0006": "The object of the invention is solved by the features of the independent claims. Further embodiments are the subject-matter of the dependent claims.", + "p0007": "BRIEF DESCRIPTION OF THE DRAWINGS", + "p0008": "The present disclosure will be described with reference to the drawings, in which:", + "p0009": "FIG. 1 illustrates an example series of communications between a customer and an archival data storage service. FIG. 2 illustrates an example environment in which archival data storage services may be implemented. FIG. 3 illustrates an interconnection network in which components of an archival data storage system may be connected. FIG. 4 illustrates an interconnection network in which components of an archival data storage system may be connected. FIG. 5 illustrates an example process for storing data. FIG. 6 illustrates an example process for retrieving data. FIG. 7 illustrates an example process for deleting data. FIG. 8 illustrates an example flow for describing an implementation of the data integrity validation described herein. FIGS. 9-10 illustrate block diagrams for describing at least some features of the data integrity validation described here. FIGS. 11-13 illustrate example flow diagrams of one or more processes for implementing at least some features of the data integrity validation described herein. FIG. 14 illustrates an example architecture for implementing data storage power management that includes an archival data storage system configured to manage operation of hard drives. FIG. 15 illustrates an additional example architecture for implementing data storage power management that includes an archival data storage system configured to manage operation of hard drives. FIGS. 16-18 illustrate example flow diagrams of processes for implementing at least some features of the data storage drive power management described herein. FIG. 19 illustrates an environment in which various embodiments can be implemented.", + "p0010": "DETAILED DESCRIPTION", + "p0011": "Techniques described and suggested herein include methods and system for providing cost-effective, reliable and scalable archival data storage services. An archival data storage system allows a customer to store, retrieve and delete archival data objects as well as perform metadata and configuration operations using a programmatic user-interface.", + "p0012": "An archival data storage service operates such that, from the perspective of users of the service data storage and deletion operations appear to be handled in a synchronous fashion while data retrieval and certain metadata operations appear to take longer time to complete. When a customer requests that a data object be stored in an archival data storage system, the system provides the customer with a data object identifier in response to the customer request. The data object identifier may be used in subsequent requests to retrieve, delete or otherwise reference the stored data object. The data object can be stored immediately in an archival data storage described herein. The data object can be stored in a transient data store. As used herein, transient data store is used interchangeably with temporary or staging data store to refer to a data store that is used to store data objects before they are stored in an archival data storage described herein or to store data objects that are retrieved from the archival data storage. A transient data store may provide volatile or non-volatile storage. While potentially usable for persistently storing data, a transient data store is intended to store data for a shorter period of time than an archival data storage system and may be less cost-effective than the data archival storage system described herein. Where data is stored in a transient data store, a data storage job may be created to eventually store the data object to the archival data storage system. At least some of the data storage job may then be processed after the data object identifier is provided to the customer. As used herein, a \"job\" refers to a data-related activity corresponding to a customer request that may be performed after the request is handled or a response is provided.", + "p0013": "When a customer requests that a data object be retrieved from an archival data storage system, the system creates a data retrieval job associated with the customer request and provides the customer with the job identifier. The request specifies a data object identifier for the data object to be retrieved, such as issued by the system when the data object is previously stored, described above. Subsequent to providing the job identifier, the system may process the data retrieval job along with other jobs in the system in order to benefit from efficiencies and other advantages of batch processing techniques. The customer may learn of the job's status by receiving a notification from the system, querying the system with the job identifier and the like, and download the retrieved data from a designated transient data store once the job is completed. Certain metadata operations are handled in a similar fashion.", + "p0014": "When a customer requests that a data object be deleted from an archival data storage system, the system provides the customer with an acknowledgement of the deletion. The data object can be deleted immediately. The system can create a data deletion job associated with the customer request and processes the data deletion job after providing the acknowledgement of deletion, possibly days or even longer after providing the acknowledgment. The system can allow a grace period for the customer to cancel, undo or otherwise recover the data object.", + "p0015": "The archival data storage system comprises a front end subsystem that handles customer requests and perform functionalities such as authentication, authorization, usage metering, accounting and billing and the like. Additionally, the front end subsystem may also handle bulk data transfers in and out of the archival data storage system.", + "p0016": "The archival data storage system can comprise a control plane for direct I/O that provides transient durable storage or staging area for incoming and outgoing payload data. In addition, the control plane for direct I/O may provide services for creating, submitting and monitoring the execution of jobs associated with customer requests.", + "p0017": "The archival data storage system can comprise a common control plane that provides a queue-based load leveling service to dampen peak to average load coming into the archival data storage system among others. The common control plane can also provide a durable and high-efficiency transient store for job execution that may be used by services in the data plane to perform job planning optimization, check pointing, recovery, and the like.", + "p0018": "The archival data storage system can comprise a data plane that provides services related to long-term archival data storage, retrieval and deletion, data management and placement, anti-entropy operations and the like. The data plane comprises a plurality of storage entities including storage devices , storage nodes, servers and the like. Storage entities are designed to provide high storage capacity and low operational costs. For example, data plane may include hard drives with Shingled Magnetic Recording technologies to offer increased hard drive capacity. As another example, a power management schedule may be used to control which hard drives are active at a given time to conserve power and cooling costs associated with hardware devices. In addition, data plane may also provide anti-entropy operations to detect entropic effects and to initiate anti-entropy routines.", + "p0019": "The archival data storage system can comprise a metadata plane that provides information about data objects stored in the system for inventory and accounting purposes, customer metadata inquiries, anti-entropy processing and the like. In particular, metadata plane may generate a cold index based on job completion records. Metadata plane services are designed to impose minimal cost overhead using techniques such as batch processing.", + "p0020": "The archival data storage system described herein is implemented to be efficient and scalable. For example, batch processing and request coalescing is used at various stages to improve efficiency. For another example, processing of metadata such as jobs, requests and the like can be partitioned so as to facilitate parallel processing of the partitions by multiple instances of services.", + "p0021": "Data elements stored in the archival data storage system are self-describing so as to avoid the need for a global index data structure. For example, data objects stored in the system may be addressable by data object identifiers that encode storage location information. For another example, volumes may store information about which data objects are stored in the volume and storage nodes and devices storing such volumes may collectively report their inventory and hardware information to provide a global view of the data stored in the system. In such a case, the global view is provided for efficiency only and not required to locate data stored in the system.", + "p0022": "The archival data storage system described herein is implemented to improve data reliability and durability. For example, a data object is redundantly encoded into a plurality of data components and stored across different data storage entities to provide fault tolerance. For another example, data elements have multiple levels of integrity checks. Parent/child relations always have additional information to ensure full referential integrity. For example, bulk data transmission and storage paths are protected by having the initiator pre-calculate the digest on the data before transmission and subsequently supply the digest with the data to a receiver. The receiver of the data transmission is responsible for recalculation, comparing and then acknowledging to the sender that includes the recalculated the digest. Such data integrity checks may be implemented, for example, by front end services, transient data storage services, data plane storage entities and the like described below in connection with FIG. 2 .", + "p0023": "Embodiments of the present disclosure are directed to validating or otherwise verifying the integrity of data payloads or portions of data payloads intended for storage. An archival data storage service may be configured to receive requests to store data, in some cases large amounts of data, in logical data containers or other archival storage devices for varying, and often relatively long, periods of time. The archival data storage service may operate or otherwise utilize many different storage devices. For example, the archival storage service may include one or more disk drives that, when operating, utilize spinning magnetic media. Additionally, the archival data storage service may include one or more racks located in one or more geographic locations . Each rack may further include one or more, or even hundreds of, hard drives such as, but not limited to, disk drives or the like.", + "p0024": "The archival data storage service may provide storage, access and/or placement of one or more computing resources through a service such as, but not limited to, a web service, a remote program execution service or other network-based data management service. For example, a user, client entity, computing resource or other computing device may access, via the archival data storage service, data storage and/or management such that access mechanisms may be implemented and/or provided to the user or computing device. In some examples, computing resource services, such as those provided by the archival data storage service, may include one or more computing resources accessible across one or more networks through user interfaces , application programming interfaces and/or other interfaces where the one or more computing resources may be scalable and/or expandable as desired.", + "p0025": "The archival data storage service may enable users or client entities such as, but not limited to, third-party services that utilize the archival data storage service or other web services associated with the archival data storage service to upload data for potentially long and persistent storage. Unless otherwise contradicted explicitly or clearly by context, the term \"user\" is used herein to describe any entity utilizing the archival data storage service. The archival data storage service may also wish to ensure the integrity of the archived data and/or guarantee the integrity of the archived data. In order to accomplish these goals, in some examples, the archival data storage service may provide a data object identifier for identifying the data once uploaded. In some cases, the data object identifier may also include a top-level tree digest for validating the archived data even after some extended period of time. Additionally, the data object identifier may also be configured in such a way that its integrity may be validated as well.", + "p0026": "The archival data storage service is be configured to perform operations on data to be stored, such that the data is broken into parts, portions or other demarcated groupings. Once separated into parts, the archival data storage service performs one or more encryption functions and/or algorithms on the parts including, but not limited to, a hash function or other cryptographic or other method to produce a digest . The data is validated such that the archival storage service can ensure that the data being stored matches the data received. In this way, data integrity may be validated. Additionally, the sender of the data may be able to independently determine a data chunk size and not be requested to maintain or persist this information. For example, a user or client entity may request to upload a 1 gigabyte file in 2 megabyte portions . And without saving or otherwise persisting the fact that 2MB chunks were used, the data payload may still be validated. That is, based at least in part on generating one or more digests, checksums, hash codes, etc., for the chunks and at least a digest for the payload , the data may later be partitioned into different sized chunks without losing the ability to be validated.", + "p0027": "This may be accomplished by providing instructions or an algorithm to a user or client entity that indicates the way in which the data should be partitioned and/or hashed. Additionally, in some cases, the archival storage service may expose or otherwise provide one or more API method calls and/or a software development kit to enable users to appropriately upload the data in chunks and to facilitate the requested order of operations and/or inclusion of appropriate checksum information. For example, the users may be requested to select a chunk size of at least 1MB, or some other predefined size, for uploading the data. As used herein, the data being uploaded by a user or client entity and/or stored by the archival data storage service may be referred to as a payload. Additionally, the user may select or otherwise instruct the archival data storage service that the payload is to be partitioned into sizes including powers of two of 1MB . In other examples, the payload may be partitioned into sizes including other multiples of a predefined size . The other multiples may be based at least in part on the degree of children of a tree used to represent the data. For example, if a binary tree is used, the integer multiple may include integer powers of two, if a trinary tree is used, the integer multiple may be integer powers of three, if a tree of degree four is used, the integer multiple may include integer powers of four, and so on. The user may then follow an algorithm for generating one or more hash trees and/or one or more hash values for each partition. Additionally, the user may generate a hash value for each 1MB chunk independent of the selected partition size. In some examples, these digests corresponding to the 1MB chunks of a partition may be included in a hash tree for the partition. Further, a root node of each partition's hash tree may be provided to the archival data storage service along with the digests for each 1MB sub-part of the partition. In this way, each sub-part may be validated by the archival data storage service, based at least in part on the 1MB digests, and each partition may be validated by the archival data storage service, based at least in part on the root digest for each part. Additionally, a final root hash associated with a hash of each root hash for the parts may be used by the archival data storage service, based at least in part on comparing the received final root hash and a top-level hash value determined by the archival data storage service.", + "p0028": "Generating a hash tree or other hierarchical data structure may include concatenating digests and running a hash function on the concatenated digest. For example, in a binary hash tree, a root node may have two children represented by one hash value each. In some cases, generating the root hash value may be based at least in part on concatenating the two children hash values to form a new piece of data and further running the hash function on the new piece of data. The resulting hash value may represent the root hash. As such, each partition of a payload may have its own root hash, although its root hash may be used in calculating the top-level hash for the payload. It may also be possible to validate the payload and/or portions of the payload without recalling or otherwise persisting the partition size chosen by the user.", + "p0029": "The present disclosure also relates to, among other things, determining and/or implementing one or more schedules for enabling, operating, providing power to or otherwise controlling one or more storage devices of a storage service. In some examples, the storage service may be an archival data storage service that operates or otherwise utilizes many different computing devices and/or storage devices. For example, the archival storage service may include one or more disk drives that, when operating, utilize spinning magnetic media. Additionally the archival data storage service may include one or more racks located in one or more geographic locations . Each rack may further include one or more, or even hundreds of hard drives such as, but not limited to, disk drives or the like. Additionally, rotating media of the hard drives, operating drive controllers and/or processors, etc., generally consume relatively large amounts of power and can be costly.", + "p0030": "An archival data storage service may be configured to minimize power and/or cooling costs associated with storage devices by changing and/or managing hard drive power consumption. By way of example only, the archival data storage service may be configured to control the number of hard drives that may have actively spinning media at a given time. Additionally, the archival data storage service may be configured to change the rotation speeds instead of shutting the drive on or off to save power. For example, some drives could be slowed down, and a rotation schedule could rotate which drives to slow down and which drives to run at normal speed. Additionally, a data storage node, which may be configured to control a data storage device such as a hard drive, may determine and/or control which hard drives are active and which hard drives are not. A hard drive may be considered active if it is spinning its media, powering up and/or powering down. For example, a spinning hard drive may include one in which the disk is rotating and a head may magnetize locations of the drive to write, read or delete data from the drive while rotating. However, a drive may have been activated; however, it may not be fully rotating yet. In this example, the drive may be powering up. Alternatively, once a drive has been activated, the data storage node or other controller may deactivate the drive; however, it may take some time before the drive is no longer spinning. This may be considered powering down. As such, there may be times when the drive is operational but not active. Further, activating a data storage device may include providing power to a device that is not currently receiving power or activating a device that is receiving power but is in a sleep mode. As such, a data storage device may be in many states including, but not limited off, on, sleep, power down or power up.", + "p0031": "The data storage nodes may be configured to determine a rotation schedule, provide the rotation schedule and/or activate and deactivate storage devices. Additionally, the storage nodes may report this information to a storage node registrar, queue or other library so that one or more storage node managers may be informed regarding which write, read or delete jobs to execute. It may be possible to execute write jobs at any time, based at least in part on an assumption that there will generally be enough capacity to handle incoming write requests and place them into whichever drives are active. However, the storage node managers may request, or otherwise be instructed, to execute write jobs in particular drives based at least in part on when they are active. In this case, the storage node managers may first request the rotation schedule, or at least an indication of which drives are active, from the storage nodes prior to executing the jobs. It also may be possible to execute write jobs at any time, based at least in part on an assumption that there will generally be enough active hard drives to process write requests at a desired rate, even though some number of drives may not be active.", + "p0032": "Rotation schedules may be based at least in part on a percentage or a number of drives to be active at the same time. For example, the archival storage service may determine that only a predefined percentage of the drives should be active at any given time. As such, all other drives may be in a sleep or other low-power state mode, while a predefined percentage may be active. Additionally, the total number of drives may be broken into groups based at least in part on the determined percentage. For example, if the archival storage service determines that 20% of the drives should be active, the set of drives to be controlled may be broken into five groups. In this way, when each group of drives is active, the predefined 20% of drives will be active. The other 80% of drives may be in a sleep or other low-power mode. Alternatively, any other percentage may be used to calculate a corresponding number of groups of drives as desired. Further, in some aspects, groups may not necessarily be equal in order to accommodate percentages that do not allow for equality among groups . Alternatively, percentages may be rounded up or down to accommodate equality among groups. Additionally, an hour may be divided by the resulting number of groups to determine how long each group should be active. For example, using the 20% example for illustrative purposes only, each group may be scheduled to be active for 12 minutes to accommodate each group being active at least once per hour. Further, in order to avoid power spikes or burnouts, drives within a group may be powered up sequentially or at least not all at the same time. Alternatively, or in addition, each drive of a group that is active may be sequentially powered down while drives of the next group to be activated are sequentially powered up.", + "p0033": "Rotation schedules may be based at least in part on a predefined number of drives to be active at a given time, as opposed to a percentage. Additionally, the rotation schedule may include a sliding window that moves at a speed or interval along a theoretical or real array of drives. Drives that are located within the window may be activated while drives outside the window may be in a low power state . As desired, the drives may be sorted on a time line based at least in part on their actual location and/or based at least in part on some logical mapping. The window may be configured to traverse the timeline. Alternatively, other examples of activating drives within a sliding window may be envisioned. The rotation schedules may also be based at least in part on demand including, but not limited to, peak power consumption times , customer demand, pending jobs, etc. By way of example only, one scenario may include scheduling drives to be actively based at least in part on pending read requests from multiple storage node managers. The storage node registrar may be configured to select a number or percentage of drives that should come into a rotation in order to meet the demand. The storage node registrar may also send a message to the storage nodes, once the drives are selected to come into the rotation, that the drives have been scheduled for rotation. Alternatively, or in addition, the storage node managers may notify the storage nodes once the registrar has included the appropriate drives in the rotation.", + "p0034": "FIG. 1 illustrates a series of communications between a customer 102 and an archival data storage system 104. The communications illustrate a series of requests and responses that may be used, for example, to store and subsequently retrieve an archival data object to and from an archival data storage system. The illustrated series of communications may occur in an environment 200 such as illustrated in FIG. 2 , described below.", + "p0035": "Archival data storage system 104 may provide a user interface for an entity such as a customer to communicate archival data storage system 104. Such a user interface may include graphical user interfaces , Web-based interfaces, programmatic interfaces such as application programming interfaces and/or sets of remote procedure calls corresponding to interface elements, messaging interfaces in which the interface elements correspond to messages of a communication protocol, and/or suitable combinations thereof. For example, a customer may use such a user interface to store, retrieve or delete data as well as to obtain metadata information, configure various operational parameters and the like. Archival data storage system 104 may include a front end service such as described in FIG. 2 below, for handling customer requests.", + "p0036": "A customer device 102 initiates a request 106 to store data, for example, using an API as describe above. The storage request 106 may include payload data such as an archival file and metadata such as a size and digest of the payload data, user identification information , an identifier of a logical storage container and the like. Multiple storage requests 106 may be used to request the upload of a large payload data, where each of the multiple storage requests may include a portion of the payload data. A storage request 106 may include multiple data objects to be uploaded.", + "p0037": "Upon receiving storage request 106, archival data storage system 104 may store 108 the data in a staging store. Only payload data may be stored in the staging storage or additional data such as a data object identifier may be stored with payload data. A job may be created for moving the data stored in staging storage to archival data storage as described herein. As used herein, a \"job\" refers to a data-related activity corresponding to a customer request that may be performed temporally independently from the time the request is received. For example, a job may include retrieving, storing and deleting data, retrieving metadata and the like. A job may be identified by a j ob identifier that may be unique, for example, among all the jobs for a particular customer, within a data center or the like. Such a job may be processed to store 114 the data object in archival data storage. The job may be executed in connection with other jobs to optimize costs, response time, efficiency, operational performance and other metrics of the system. For example, jobs may be sorted by storage location, coalesced, batch-processed or otherwise optimized to improve throughput, reduce power consumption and the like. As another example, jobs may be partitioned and each partition may be processed in parallel by a different instance of service.", + "p0038": "Archival data storage system 104 sends a response 110 to customer 102 acknowledging the storage of data . Thus, from the customer's perspective, the storage request is handled in a synchronous fashion. The response can include a data object identifier that may be used by subsequent customer requests to retrieve, delete or otherwise manage the data. Customer 102 may store 112 the data object identifier in a customer-side data store, a data storage service and the like. For example, customer 102 may maintain a map associating data object identifiers with corresponding user-friendly names or global unique identifiers . Storage requests may be perceived by a customer as being handled in an asynchronous fashion in a manner similar to that described below in connection with retrieval requests.", + "p0039": "Data object identifiers may encode storage location information that may be used to locate the stored data object, payload validation information such as size, timestamp, digest and the like that may be used to validate the integrity of the payload data, metadata validation information such as error-detection codes that may be used to validate the integrity of metadata such as the data object identifier itself, policy information that may be used to validate the requested access and the like. Alternatively, data object identifier may include the above information without any encoding such as encryption.", + "p0040": "A customer 102 initiates a request 116 to retrieve data stored in archival data storage system 104, for example, using an API described above. The retrieval request 116 may specify a data object identifier associated the data to be retrieved, such as provided in response 110, described above. Upon receiving retrieval request 116, archival data storage system 104 creates 118 a retrieval job to retrieve data from an archival data storage described herein. The system may create such a retrieval job in a manner similar to that described in connection of a storage job, above.", + "p0041": "Archival data storage system 104 provides a response 120 with the job identifier for the retrieval job. Thus, from a customer's perspective, the retrieval request is handled in an asynchronous fashion. Customer 102 can store 122 the job identifier in a customer-side storage and may optionally use the job identifier query or poll archival data storage system for the status of the retrieval job.", + "p0042": "Archival data storage system 104 processes 124 the retrieval job in connection with other jobs using techniques such as batch processing to optimize costs. After the processing of the retrieval job, archival data storage system 104 stores 126 the retrieved data in a staging store similar to the staging store for storing data associated with storage requests, discussed above.", + "p0043": "Archival data storage system 104 provides a notification 128 to customer 102 after the requested data is retrieved, for example, in a staging store. Such notifications may be configurable by customers and may include job identifier associated with the retrieval request, a data object identifier, a path to a download location, identity of the customer or any other information that is used to download the retrieved data. Such notifications may be provided by a notification service that may or may not be part of the archival data storage system. Customer 102 may query or poll the archival data storage system with the retrieval job identifier regarding the status of the job.", + "p0044": "Upon learning of the completion of the retrieval job, customer 102 sends a download request 130 to download the staged data. Download request 130 may provide information included in the notification such as job identifier, data object identifier and other identifying information such as customer account identifier and the like that would be used to download the staged data. In response to the download request 130, archival data storage system 104 may retrieve 132 the staged data from the staging store and provide the retrieved data to customer 102 in a response 134.", + "p0045": "FIG. 2 illustrates an example environment 200 in which an archival data storage system may be implemented. One or more customers 202 connect, via a network 204, to an archival data storage system 206. As implied above, unless otherwise clear from context, the term \"customer\" refers to the system of a customer entity that utilizes data storage services described herein. Such systems may include datacenters, mainframes, individual computing devices, distributed computing environments and customer-accessible instances thereof or any other system capable of communicating with the archival data storage system. A customer may refer to a machine instance or virtual instance of a distributed computing system provided by a computing resource provider that also provides the archival data storage system. The archival data storage system can be integral to the distributed computing system and may include or be implemented by an instance, virtual or machine, of the distributed computing system. Network 204 may include the Internet, a local area network , a wide area network , a cellular data network and/or other data network.", + "p0046": "Archival data storage system 206 provides a multi-tenant or multi-customer environment where each tenant or customer may store, retrieve, delete or otherwise manage data in a data storage space allocated to the customer. An archival data storage system 206 comprises multiple subsystems or \"planes\" that each provides a particular set of services or functionalities. For example, as illustrated in FIG. 2 , archival data storage system 206 includes front end 208, control plane for direct I/O 210, common control plane 212, data plane 214 and metadata plane 216. Each subsystem or plane may comprise one or more components that collectively provide the particular set of functionalities. Each component may be implemented by one or more physical and/or logical computing devices, such as computers, data storage devices and the like. Components within each subsystem may communicate with components within the same subsystem, components in other subsystems or external entities such as customers. At least some of such interactions are indicated by arrows in FIG. 2 . In particular, the main bulk data transfer paths in and out of archival data storage system 206 are denoted by bold arrows. It will be appreciated by those of ordinary skill in the art that there may be fewer or a greater number of systems, subsystems and/or subcomponents than are illustrated in FIG. 2 . Thus, the depiction of environment 200 in FIG. 2 should be taken as being illustrative in nature.", + "p0047": "In FIG. 2 , front end 208 implements a group of services that provides an interface between the archival data storage system 206 and external entities, such as one or more customers 202 described herein. Front end 208 provides an application programming interface to enable a user to programmatically interface with the various features, components and capabilities of the archival data storage system. Such APIs may be part of a user interface that may include graphical user interfaces , Web-based interfaces, programmatic interfaces such as application programming interfaces and/or sets of remote procedure calls corresponding to interface elements, messaging interfaces in which the interface elements correspond to messages of a communication protocol, and/or suitable combinations thereof.", + "p0048": "Capabilities provided by archival data storage system 206 may include data storage, data retrieval, data deletion, metadata operations, configuration of various operational parameters and the like. Metadata operations may include requests to retrieve catalogs of data stored for a particular customer, data recovery requests, job inquires and the like. Configuration APIs may allow customers to configure account information, audit logs, policies, notifications settings and the like. A customer may request the performance of any of the above operations by sending API requests to the archival data storage system. Similarly, the archival data storage system may provide responses to customer requests. Such requests and responses may be submitted over any suitable communications protocol, such as Hypertext Transfer Protocol , File Transfer Protocol and the like, in any suitable format, such as REpresentational State Transfer , Simple Object Access Protocol and the like. The requests and responses may be encoded, for example, using Base64 encoding, encrypted with a cryptographic key or the like.", + "p0049": "Archival data storage system 206 allows customers to create one or more logical structures such as a logical data containers in which to store one or more archival data objects. As used herein, data object is used broadly and does not necessarily imply any particular structure or relationship to other data. A data object may be, for instance, simply a sequence of bits. Typically, such logical data structures may be created to meeting certain business requirements of the customers and are independently of the physical organization of data stored in the archival data storage system. As used herein, the term \"logical data container\" refers to a grouping of data objects. For example, data objects created for a specific purpose or during a specific period of time may be stored in the same logical data container. Each logical data container may include nested data containers or data objects and may be associated with a set of policies such as size limit of the container, maximum number of data objects that may be stored in the container, expiration date, access control list and the like. Logical data containers may be created, deleted or otherwise modified by customers via API requests, by a system administrator or by the data storage system, for example, based on configurable information. For example, the following HTTP PUT request may be used to create a logical data container with name \"logical-container-name\" associated with a customer identified by an account identifier \"accountId\".PUT / /logical-container-name HTTP/1.1", + "p0050": "Archival data storage system 206 provides the APIs for customers to store data objects into logical data containers. For example, the following HTTP POST request may be used to store a data object into a given logical container. The request may specify the logical path of the storage location, data length, reference to the data payload, a digital digest of the data payload and other information. The APIs may allow a customer to upload multiple data objects to one or more logical data containers in one request. Where the data object is large, the APIs may allow a customer to upload the data object in multiple parts, each with a portion of the data object.POST / /logical-container-name/data HTTP/1.1Content-Length: 1128192x-ABC-data-description: \"annual-result-2012.xls\"x-ABC-md5-tree-hash: 634d9a0688aff95c", + "p0051": "In response to a data storage request, archival data storage system 206 provides a data object identifier if the data object is stored successfully. Such data object identifier may be used to retrieve, delete or otherwise refer to the stored data object in subsequent requests. Such as data object identifier may be \"self-describing\" in that it includes storage location information that may be used by the archival data storage system to locate the data object without the need for an additional data structures such as a global namespace key map. In addition, data object identifiers may also encode other information such as payload digest, error-detection code, access control data and the other information that may be used to validate subsequent requests and data integrity. The archival data storage system stores incoming data in a transient durable data store before moving it archival data storage. Thus, although customers may perceive that data is persisted durably at the moment when an upload request is completed, actual storage to a long-term persisted data store may not commence until sometime later . The timing of the actual storage may depend on the size of the data object, the system load during a diurnal cycle, configurable information such as a service-level agreement between a customer and a storage service provider and other factors.", + "p0052": "Archival data storage system 206 provides the APIs for customers to retrieve data stored in the archival data storage system. A customer may initiate a job to perform the data retrieval and may learn the completion of the job by a notification or by polling the system for the status of the job. As used herein, a \"job\" refers to a data-related activity corresponding to a customer request that may be performed temporally independently from the time the request is received. For example, a job may include retrieving, storing and deleting data, retrieving metadata and the like. A job may be identified by a job identifier that may be unique, for example, among all the jobs for a particular customer. For example, the following HTTP POST request may be used to initiate a job to retrieve a data object identified by a data object identifier \"dataObjectId.\" A data retrieval request may request the retrieval of multiple data objects, data objects associated with a logical data container and the like.POST / /logical-data-container-name/data/ HTTP/1.1", + "p0053": "In response to the request, archival data storage system 206 provides a job identifier job-id,\" that is assigned to the job in the following response. The response provides, in this example, a path to the storage location where the retrieved data will be stored.HTTP/1.1 202 ACCEPTEDLocation: / /logical-data-container-name/jobs/", + "p0054": "At any given point in time, the archival data storage system may have many jobs pending for various data operations. The archival data storage system may employ job planning and optimization techniques such as batch processing, load balancing, job coalescence and the like, to optimize system metrics such as cost, performance, scalability and the like. The timing of the actual data retrieval depends on factors such as the size of the retrieved data, the system load and capacity, active status of storage devices and the like. For example, at least some data storage devices in an archival data storage system may be activated or inactivated according to a power management schedule, for example, to reduce operational costs. Thus, retrieval of data stored in a currently active storage device may be faster than retrieval of data stored in a currently inactive storage device .", + "p0055": "When a data retrieval job is completed, the retrieved data is stored in a staging data store and made available for customer download. A customer is notified of the change in status of a job by a configurable notification service. A customer may learn of the status of a job by polling the system using a job identifier. The following HTTP GET request may be used to download data that is retrieved by a job identified by \"job-id,\" using a download path that has been previously provided.GET / /logical-data-container-name/jobs/ /outputHTTP/1.1", + "p0056": "In response to the GET request, archival data storage system 206 may provide the retrieved data in the following HTTP response, with a tree-hash of the data for verification purposes.HTTP/1.1 200 OKContent-Length: 1128192x-ABC-archive-description: \"retrieved stuff\"x-ABC-md5-tree-hash: 693d9a7838aff95c", + "p0057": "A customer may request the deletion of a data object stored in an archival data storage system by specifying a data object identifier associated with the data object. For example, a data object with data object identifier \"dataObjectId\" may be deleted using the following HTTP request. A customer may request the deletion of multiple data objects such as those associated with a particular logical data container.DELETE / /logical-data-container-name/data/ HTTP/1.1", + "p0058": "Data objects may be deleted in response to a customer request or may be deleted automatically according to a user-specified or default expiration date. Data objects may be rendered inaccessible to customers upon an expiration time but remain recoverable during a grace period beyond the expiration time. The grace period may be based on configurable information such as customer configuration, service-level agreement terms and the like. A customer may be provided the abilities to query or receive notifications for pending data deletions and/or cancel one or more of the pending data deletions. For example, a customer may set up notification configurations associated with a logical data container such that the customer will receive notifications of certain events pertinent to the logical data container. Such events may include the completion of a data retrieval job request, the completion of metadata request, deletion of data objects or logical data containers and the like.", + "p0059": "Archival data storage system 206 also provides metadata APIs for retrieving and managing metadata such as metadata associated with logical data containers. Such requests may be handled asynchronously or synchronously .", + "p0060": "Still referring to FIG. 2 , at least some of the API requests discussed above are handled by API request handler 218 as part of front end 208. For example, API request handler 218 may decode and/or parse an incoming API request to extract information, such as uniform resource identifier , requested action and associated parameters, identity information, data object identifiers and the like. In addition, API request handler 218 invoke other services , where necessary, to further process the API request.", + "p0061": "Front end 208 includes an authentication service 220 that may be invoked, for example, by API handler 218, to authenticate an API request. For example, authentication service 220 may verify identity information submitted with the API request such as username and password Internet Protocol address, cookies, digital certificate, digital signature and the like. Authentication service 220 may require the customer to provide additional information or perform additional steps to authenticate the request, such as required in a multifactor authentication scheme, under a challenge-response authentication protocol and the like.", + "p0062": "Front end 208 includes an authorization service 222 that may be invoked, for example, by API handler 218, to determine whether a requested access is permitted according to one or more policies determined to be relevant to the request. For example, authorization service 222 verifies that a requested access is directed to data objects contained in the requestor's own logical data containers or which the requester is otherwise authorized to access. Authorization service 222 or other services of front end 208 may check the validity and integrity of a data request based at least in part on information encoded in the request, such as validation information encoded by a data object identifier.", + "p0063": "Front end 208 includes a metering service 224 that monitors service usage information for each customer such as data storage space used, number of data objects stored, data requests processed and the like. Front end 208 also includes accounting service 226 that performs accounting and billing-related functionalities based, for example, on the metering information collected by the metering service 224, customer account information and the like. For example, a customer may be charged a fee based on the storage space used by the customer, size and number of the data objects, types and number of requests submitted, customer account type, service level agreement the like.", + "p0064": "Front end 208 batch processes some or all incoming requests. For example, front end 208 may wait until a certain number of requests has been received before processing the requests. Such a batch processing of incoming requests may be used to gain efficiency.", + "p0065": "Front end 208 may invoke services provided by other subsystems of the archival data storage system to further process an API request. For example, front end 208 may invoke services in metadata plane 216 to fulfill metadata requests. For another example, front end 208 may stream data in and out of control plane for direct I/O 210 for data storage and retrieval requests, respectively.", + "p0066": "Referring now to control plane for direct I/O 210 illustrated in FIG. 2 , control plane for direct I/O 210 provides services that create, track and manage jobs created as a result of customer requests. As discussed above, a job refers to a customer-initiated activity that may be performed asynchronously to the initiating request, such as data retrieval, storage, metadata queries or the like. Control plane for direct I/O 210 includes a job tracker 230 that is configured to create job records or entries corresponding to customer requests, such as those received from API request handler 218, and monitor the execution of the jobs. A job record may include information related to the execution of a job such as a customer account identifier, job identifier, data object identifier, reference to payload data cache 228 , job status, data validation information and the like. Job tracker 230 may collect information necessary to construct a job record from multiple requests. For example, when a large amount of data is requested to be stored, data upload may be broken into multiple requests, each uploading a portion of the data. In such a case, job tracker 230 may maintain information to keep track of the upload status to ensure that all data parts have been received before a job record is created. Job tracker 230 also obtains a data object identifier associated with the data to be stored and provides the data object identifier, for example, to a front end service to be returned to a customer. Such data object identifier may be obtained from data plane 214 services such as storage node manager 244, storage node registrar 248, and the like, described below.", + "p0067": "Control plane for direct I/O 210 includes a job tracker store 232 for storing job entries or records. Job tracker store 230 may be implemented by a NoSQL data management system, such as a key-value data store, a relational database management system or any other data storage system. Data stored in job tracker store 230 may be partitioned to enable fast enumeration of jobs that belong to a specific customer, facilitate efficient bulk record deletion, parallel processing by separate instances of a service and the like. For example, job tracker store 230 may implement tables that are partitioned according to customer account identifiers and that use job identifiers as range keys. Job tracker store 230 is further sub-partitioned based on time to facilitate job expiration and cleanup operations. Transactions against job tracker store 232 may be aggregated to reduce the total number of transactions. For example, a job tracker 230 may perform aggregate multiple jobs corresponding to multiple requests into one single aggregated job before inserting it into job tracker store 232.", + "p0068": "Job tracker 230 may be configured to submit the job for further job scheduling and planning, for example, by services in common control plane 212. Additionally, job tracker 230 may be configured to monitor the execution of jobs and update corresponding job records in job tracker store 232 as jobs are completed. Job tracker 230 may be further configured to handle customer queries such as job status queries. Job tracker 230 may also provide notifications of job status changes to customers or other services of the archival data storage system. For example, when a data retrieval job is completed, job tracker 230 may cause a customer to be notified that data is available for download. As another example, when a data storage job is completed, job tracker 230 may notify a cleanup agent 234 to remove payload data associated with the data storage job from a transient payload data cache 228, described below.", + "p0069": "Control plane for direct I/O 210 includes a payload data cache 228 for providing transient data storage services for payload data transiting between data plane 214 and front end 208. Such data includes incoming data pending storage and outgoing data pending customer download. As used herein, transient data store is used interchangeably with temporary or staging data store to refer to a data store that is used to store data objects before they are stored in an archival data storage described herein or to store data objects that are retrieved from the archival data storage. A transient data store may provide volatile or non-volatile storage. While potentially usable for persistently storing data, a transient data store is intended to store data for a shorter period of time than an archival data storage system and may be less cost-effective than the data archival storage system described herein. Transient data storage services provided for incoming and outgoing data may be differentiated. For example, data storage for the incoming data, which is not yet persisted in archival data storage, may provide higher reliability and durability than data storage for outgoing data, which is already persisted in archival data storage. Transient storage may be optional for incoming data, that is, incoming data may be stored directly in archival data storage without being stored in transient data storage such as payload data cache 228, for example, when there is the system has sufficient bandwidth and/or capacity to do so.", + "p0070": "Control plane for direct I/O 210 also includes a cleanup agent 234 that monitors job tracker store 232 and/or payload data cache 228 and removes data that is no longer needed. For example, payload data associated with a data storage request may be safely removed from payload data cache 228 after the data is persisted in permanent storage . On the reverse path, data staged for customer download may be removed from payload data cache 228 after a configurable period of time or after a customer indicates that the staged data is no longer needed.", + "p0071": "Cleanup agent 234 removes a job record from job tracker store 232 when the job status indicates that the job is complete or aborted. As discussed above, job tracker store 232 may be partitioned to enable to enable faster cleanup. Where data is partitioned by customer account identifiers, cleanup agent 234 may remove an entire table that stores jobs for a particular customer account when the jobs are completed instead of deleting individual jobs one at a time. Where data is further sub-partitioned based on job expiration time cleanup agent 234 may bulk-delete a whole partition or table of jobs after all the jobs in the partition expire. Cleanup agent 234 may receive instructions or control messages from other services such as job tracker 230 that cause the cleanup agent 234 to remove job records from job tracker store 232 and/or payload data cache 228.", + "p0072": "Referring now to common control plane 212 illustrated in FIG. 2 . Common control plane 212 provides a queue-based load leveling service to dampen peak to average load levels coming from control plane for I/O 210 and to deliver manageable workload to data plane 214. Common control plane 212 includes a job request queue 236 for receiving jobs created by job tracker 230 in control plane for direct I/O 210, described above, a storage node manager job store 240 from which services from data plane 214 pick up work to execute and a request balancer 238 for transferring job items from job request queue 236 to storage node manager job store 240 in an intelligent manner.", + "p0073": "Job request queue 236 provides a service for inserting items into and removing items from a queue or first-in-last-out ), a set or any other suitable data structure. Job entries in the job request queue 236 may be similar to or different from job records stored in job tracker store 232, described above.", + "p0074": "Common control plane 212 also provides a durable high efficiency job store, storage node manager job store 240, that allows services from data plane 214 to perform job planning optimization, check pointing and recovery. For example, storage node manager job store 240 allows the job optimization such as batch processing, operation coalescing and the like by supporting scanning, querying, sorting or otherwise manipulating and managing job items stored in storage node manager job store 240. A storage node manager 244 scans incoming jobs and sort the jobs by the type of data operation , storage locations , customer account identifier and the like. The storage node manager 244 may then reorder, coalesce, group in batches or otherwise manipulate and schedule the jobs for processing. For example, the storage node manager 244 may batch process all the write operations before all the read and delete operations. The storage node manager 224 may perform operation coalescing. For another example, the storage node manager 224 may coalesce multiple retrieval jobs for the same object into one job or cancel a storage job and a deletion job for the same data object where the deletion job comes after the storage job.", + "p0075": "Storage node manager job store 240 is partitioned, for example, based on job identifiers, so as to allow independent processing of multiple storage node managers 244 and to provide even distribution of the incoming workload to all participating storage node managers 244. Storage node manager job store 240 may be implemented by a NoSQL data management system, such as a key-value data store, a RDBMS or any other data storage system.", + "p0076": "Request balancer 238 provides a service for transferring job items from job request queue 236 to storage node manager job store 240 so as to smooth out variation in workload and to increase system availability. For example, request balancer 238 may transfer job items from job request queue 236 at a lower rate or at a smaller granularity when there is a surge in job requests coming into the job request queue 236 and vice versa when there is a lull in incoming job requests so as to maintain a relatively sustainable level of workload in the storage node manager store 240. Such sustainable level of workload is around the same or below the average workload of the system.", + "p0077": "Job items that are completed are removed from storage node manager job store 240 and added to the job result queue 242. Data plane 214 services are responsible for removing the job items from the storage node manager job store 240 and adding them to job result queue 242. Job request queue 242 is implemented in a similar manner as job request queue 235, discussed above.", + "p0078": "Referring now to data plane 214 illustrated in FIG. 2 . Data plane 214 provides services related to long-term archival data storage, retrieval and deletion, data management and placement, anti-entropy operations and the like. Data plane 214 may include any number and type of storage entities such as data storage devices , storage nodes or servers, datacenters and the like. Such storage entities may be physical, virtual or any abstraction thereof and may be organized into any topology, including hierarchical or tiered topologies. Similarly, the components of the data plane may be dispersed, local or any combination thereof. For example, various computing or storage components may be local or remote to any number of datacenters, servers or data storage devices, which in turn may be local or remote relative to one another. Physical storage entities may be designed for minimizing power and cooling costs by controlling the portions of physical hardware that are active . Physical storage entities implement techniques, such as Shingled Magnetic Recording , to increase storage capacity.", + "p0079": "In an environment illustrated by FIG. 2 , one or more storage node managers 244 each controls one or more storage nodes 246 by sending and receiving data and control messages. Each storage node 246 in turn controls a collection of data storage devices such as hard disk drives. A storage node manager 244 may communicate with one or more storage nodes 246 and a storage node 246 may communicate with one or more storage node managers 244. Storage node managers 244 are implemented by one or more computing devices that are capable of performing relatively complex computations such as digest computation, data encoding and decoding, job planning and optimization and the like. Storage nodes 244 are implemented by one or more computing devices with less powerful computation capabilities than storage node managers 244. Further, the storage node manager 244 may not be included in the data path. For example, data may be transmitted from the payload data cache 228 directly to the storage nodes 246 or from one or more storage nodes 246 to the payload data cache 228. In this way, the storage node manager 244 may transmit instructions to the payload data cache 228 and/or the storage nodes 246 without receiving the payloads directly from the payload data cache 228 and/or storage nodes 246. A storage node manager 244 may send instructions or control messages to any other components of the archival data storage system 206 described herein to direct the flow of data.", + "p0080": "A storage node manager 244 serves as an entry point for jobs coming into and out of data plane 214 by picking job items from common control plane 212 , retrieving staged data from payload data cache 228 and performing necessary data encoding for data storage jobs and requesting appropriate storage nodes 246 to store, retrieve or delete data. Once the storage nodes 246 finish performing the requested data operations, the storage node manager 244 may perform additional processing, such as data decoding and storing retrieved data in payload data cache 228 for data retrieval jobs, and update job records in common control plane 212 .", + "p0081": "Storage node manager 244 performs data encoding according to one or more data encoding schemes before data storage to provide data redundancy, security and the like. Such data encoding schemes may include encryption schemes, redundancy encoding schemes such as erasure encoding, redundant array of independent disks encoding schemes, replication and the like. Likewise, storage node managers 244 performs corresponding data decoding schemes, such as decryption, erasure-decoding and the like, after data retrieval to restore the original data.", + "p0082": "As discussed above in connection with storage node manager job store 240, storage node managers 244 may implement job planning and optimizations such as batch processing, operation coalescing and the like to increase efficiency. Jobs are partitioned among storage node managers so that there is little or no overlap between the partitions. This facilitates parallel processing by multiple storage node managers, for example, by reducing the probability of racing or locking.", + "p0083": "Data plane 214 is implemented to facilitate data integrity. For example, storage entities handling bulk data flows such as storage nodes managers 244 and/or storage nodes 246 may validate the digest of data stored or retrieved, check the error-detection code to ensure integrity of metadata and the like.", + "p0084": "Data plane 214 is implemented to facilitate scalability and reliability of the archival data storage system. For example, storage node managers 244 maintain no or little internal state so that they can be added, removed or replaced with little adverse impact. Each storage device is a self-contained and self-describing storage unit capable of providing information about data stored thereon. Such information may be used to facilitate data recovery in case of data loss. Furthermore, each storage node 246 is capable of collecting and reporting information about the storage node including the network location of the storage node and storage information of connected storage devices to one or more storage node registrars 248 and/or storage node registrar stores 250. Storage nodes 246 perform such self-reporting at system start up time and periodically provide updated information. Such a self-reporting approach provides dynamic and up-to-date directory information without the need to maintain a global namespace key map or index which can grow substantially as large amounts of data objects are stored in the archival data system.", + "p0085": "Data plane 214 may also include one or more storage node registrars 248 that provide directory information for storage entities and data stored thereon, data placement services and the like. Storage node registrars 248 may communicate with and act as a front end service to one or more storage node registrar stores 250, which provide storage for the storage node registrars 248. Storage node registrar store 250 may be implemented by a NoSQL data management system, such as a key-value data store, a RDBMS or any other data storage system. Storage node registrar stores 250 may be partitioned to enable parallel processing by multiple instances of services. As discussed above, information stored at storage node registrar store 250 is based at least partially on information reported by storage nodes 246 themselves.", + "p0086": "Storage node registrars 248 provide directory service, for example, to storage node managers 244 that want to determine which storage nodes 246 to contact for data storage, retrieval and deletion operations. For example, given a volume identifier provided by a storage node manager 244, storage node registrars 248 may provide, based on a mapping maintained in a storage node registrar store 250, a list of storage nodes that host volume components corresponding to the volume identifier. Specifically, storage node registrar store 250 stores a mapping between a list of identifiers of volumes or volume components and endpoints, such as Domain Name System names, of storage nodes that host the volumes or volume components.", + "p0087": "As used herein, a \"volume\" refers to a logical storage space within a data storage system in which data objects may be stored. A volume may be identified by a volume identifier. A volume may reside in one physical storage device or span across multiple storage devices. In the latter case, a volume comprises a plurality of volume components each residing on a different storage device. As used herein, a \"volume component\" refers a portion of a volume that is physically stored in a storage entity such as a storage device. Volume components for the same volume may be stored on different storage entities. When data is encoded by a redundancy encoding scheme , each encoded data component or \"shard\" may be stored in a different volume component to provide fault tolerance and isolation. A volume component is identified by a volume component identifier that includes a volume identifier and a shard slot identifier. As used herein, a shard slot identifies a particular shard, row or stripe of data in a redundancy encoding scheme. For example, a shard slot corresponds to an erasure coding matrix row. Storage node registrar store 250 also stores information about volumes or volume components such as total, used and free space, number of data objects stored and the like.", + "p0088": "Data plane 214 also includes a storage allocator 256 for allocating storage space on storage nodes to store new data objects, based at least in part on information maintained by storage node registrar store 250, to satisfy data isolation and fault tolerance constraints. Storage allocator 256 requires manual intervention.", + "p0089": "Data plane 214 also includes an anti-entropy watcher 252 for detecting entropic effects and initiating anti-entropy correction routines. For example, anti-entropy watcher 252 may be responsible for monitoring activities and status of all storage entities such as storage nodes, reconciling live or actual data with maintained data and the like. Entropic effects include, but are not limited to, performance degradation due to data fragmentation resulting from repeated write and rewrite cycles, hardware wear , data unavailability and/or data loss due to hardware/software malfunction, environmental factors, physical destruction of hardware, random chance or other causes. Anti-entropy watcher 252 may detect such effects and may preemptively and/or reactively institute anti-entropy correction routines and/or policies.", + "p0090": "Anti-entropy watcher 252 causes storage nodes 246 to perform periodic anti-entropy scans on storage devices connected to the storage nodes. Anti-entropy watcher 252 may also inject requests in job request queue 236 to collect information, recover data and the like. Anti-entropy watcher 252 may perform scans, for example, on cold index store 262, described below, and storage nodes 246, to ensure referential integrity.", + "p0091": "Information stored at storage node registrar store 250 is used by a variety of services such as storage node registrar 248, storage allocator 256, anti-entropy watcher 252 and the like. For example, storage node registrar 248 may provide data location and placement services during data storage, retrieval and deletion. For example, given the size of a data object to be stored and information maintained by storage node registrar store 250, a storage node registrar 248 may determine where to store the data object and provides an indication of the storage location of the data object which may be used to generate a data object identifier associated with the data object. As another example, storage allocator 256 uses information stored in storage node registrar store 250 to create and place volume components for new volumes in specific storage nodes to satisfy isolation and fault tolerance constraints. As yet another example, anti-entropy watcher 252 uses information stored in storage node registrar store 250 to detect entropic effects such as data loss, hardware failure and the like.", + "p0092": "Data plane 214 also includes an orphan cleanup data store 254, which is used to track orphans in the storage system. As used herein, an orphan is a stored data object that is not referenced by any external entity. Orphan cleanup data store 254 may be implemented by a NoSQL data management system, such as a key-value data store, an RDBMS or any other data storage system. Storage node registrars 248 stores object placement information in orphan cleanup data store 254. Subsequently, information stored in orphan cleanup data store 254 may be compared, for example, by an anti-entropy watcher 252, with information maintained in metadata plane 216. If an orphan is detected, a request is inserted in the common control plane 212 to delete the orphan.", + "p0093": "Referring now to metadata plane 216 illustrated in FIG. 2 . Metadata plane 216 provides information about data objects stored in the system for inventory and accounting purposes, to satisfy customer metadata inquiries and the like. Metadata plane 216 includes a metadata manager job store 258 which stores information about executed transactions based on entries from job result queue 242 in common control plane 212. Metadata manager job store 258 may be implemented by a NoSQL data management system, such as a key-value data store, a RDBMS or any other data storage system. Metadata manager job store 258 is partitioned and sub-partitioned, for example, based on logical data containers, to facilitate parallel processing by multiple instances of services such as metadata manager 260.", + "p0094": "Metadata plane 216 also includes one or more metadata managers 260 for generating a cold index of data objects based on records in metadata manager job store 258. As used herein, a \"cold\" index refers to an index that is updated infrequently. A cold index is maintained to reduce cost overhead. Multiple metadata managers 260 may periodically read and process records from different partitions in metadata manager job store 258 in parallel and store the result in a cold index store 262.", + "p0095": "Cold index store 262 may be implemented by a reliable and durable data storage service. Cold index store 262 is configured to handle metadata requests initiated by customers. For example, a customer may issue a request to list all data objects contained in a given logical data container. In response to such a request, cold index store 262 may provide a list of identifiers of all data objects contained in the logical data container based on information maintained by cold index 262. An operation may take a relative long period of time and the customer may be provided a job identifier to retrieve the result when the job is done. Cold index store 262 is configured to handle inquiries from other services, for example, from front end 208 for inventory, accounting and billing purposes.", + "p0096": "Metadata plane 216 may also include a container metadata store 264 that stores information about logical data containers such as container ownership, policies, usage and the like. Such information may be used, for example, by front end 208 services, to perform authorization, metering, accounting and the like. Container metadata store 264 may be implemented by a NoSQL data management system, such as a key-value data store, a RDBMS or any other data storage system.", + "p0097": "As described herein, the archival data storage system 206 described herein is implemented to be efficient and scalable. For example, batch processing and request coalescing is used at various stages to improve efficiency. For another example, processing of metadata such as jobs, requests and the like are partitioned so as to facilitate parallel processing of the partitions by multiple instances of services.", + "p0098": "Data elements stored in the archival data storage system are self-describing so as to avoid the need for a global index data structure. For example, data objects stored in the system may be addressable by data object identifiers that encode storage location information. For another example, volumes may store information about which data objects are stored in the volume and storage nodes and devices storing such volumes may collectively report their inventory and hardware information to provide a global view of the data stored in the system . The global view is provided for efficiency only and not required to locate data stored in the system.", + "p0099": "The archival data storage system described herein is implemented to improve data reliability and durability. For example, a data object is redundantly encoded into a plurality of data components and stored across different data storage entities to provide fault tolerance. For another example, data elements have multiple levels of integrity checks. Parent/child relations always have additional information to ensure full referential integrity. For example, bulk data transmission and storage paths are protected by having the initiator pre-calculate the digest on the data before transmission and subsequently supply the digest with the data to a receiver. The receiver of the data transmission is responsible for recalculation, comparing and then acknowledging to the sender that includes the recalculated the digest. Such data integrity checks may be implemented, for example, by front end services, transient data storage services, data plane storage entities and the like described above.", + "p0100": "FIG. 3 illustrates an interconnection network 300 in which components of an archival data storage system may be connected. In particular, the illustrated example shows how data plane components are connected to the interconnection network 300. The interconnection network 300 may include a fat tree interconnection network where the link bandwidth grows higher or \"fatter\" towards the root of the tree. In the illustrated example, data plane includes one or more datacenters 301. Each datacenter 301 may include one or more storage node manager server racks 302 where each server rack hosts one or more servers that collectively provide the functionality of a storage node manager such as described in connection with FIG. 2 . Each storage node manager server rack may host more than one storage node manager. Configuration parameters such as number of storage node managers per rack, number of storage node manager racks and the like may be determined based on factors such as cost, scalability, redundancy and performance requirements, hardware and software resources and the like.", + "p0101": "Each storage node manager server rack 302 may have a storage node manager rack connection 314 to an interconnect 308 used to connect to the interconnection network 300. The connection 314 is implemented using a network switch 303 that may include a top-of-rack Ethernet switch or any other type of network switch. Interconnect 308 is used to enable high-bandwidth and low-latency bulk data transfers. For example, interconnect may include a Clos network, a fat tree interconnect, an Asynchronous Transfer Mode network, a Fast or Gigabit Ethernet and the like.", + "p0102": "The bandwidth of storage node manager rack connection 314 may be configured to enable high-bandwidth and low-latency communications between storage node managers and storage nodes located within the same or different data centers. For example, the storage node manager rack connection 314 has a bandwidth of 10 Gigabit per second .", + "p0103": "Each datacenter 301 may also include one or more storage node server racks 304 where each server rack hosts one or more servers that collectively provide the functionalities of a number of storage nodes such as described in connection with FIG. 2 . Configuration parameters such as number of storage nodes per rack, number of storage node racks, ration between storage node managers and storage nodes and the like may be determined based on factors such as cost, scalability, redundancy and performance requirements, hardware and software resources and the like. For example, there are 3 storage nodes per storage node server rack, 30-80 racks per data center and a storage nodes / storage node manager ratio of 10 to 1.", + "p0104": "Each storage node server rack 304 may have a storage node rack connection 316 to an interconnection network switch 308 used to connect to the interconnection network 300. The connection 316 is implemented using a network switch 305 that may include a top-of-rack Ethernet switch or any other type of network switch. The bandwidth of storage node rack connection 316 may be configured to enable high-bandwidth and low-latency communications between storage node managers and storage nodes located within the same or different data centers. A storage node rack connection 316 has a higher bandwidth than a storage node manager rack connection 314. For example, the storage node rack connection 316 has a bandwidth of 20 Gbps while a storage node manager rack connection 314 has a bandwidth of 10 Gbps.", + "p0105": "Datacenters 301 communicate, via connection 310, with other computing resources services 306 such as payload data cache 228, storage node manager job store 240, storage node registrar 248, storage node registrar store 350, orphan cleanup data store 254, metadata manager job store 258 and the like as described in connection with FIG. 2 .", + "p0106": "One or more datacenters 301 may be connected via inter-datacenter connection 312. Connections 310 and 312 may be configured to achieve effective operations and use of hardware resources. For example, connection 310 has a bandwidth of 30-100 Gbps per datacenter and inter-datacenter connection 312 has a bandwidth of 100-250 Gbps.", + "p0107": "FIG. 4 illustrates an interconnection network 400 in which components of an archival data storage system may be connected. In particular, the illustrated example shows how non-data plane components are connected to the interconnection network 300. As illustrated, front end services, such as described in connection with FIG. 2 , may be hosted by one or more front end server racks 402. For example, each front end server rack 402 may host one or more web servers. The front end server racks 402 may be connected to the interconnection network 400 via a network switch 408. Configuration parameters such as number of front end services, number of services per rack, bandwidth for front end server rack connection 314 and the like may roughly correspond to those for storage node managers as described in connection with FIG. 3 .", + "p0108": "Control plane services and metadata plane services as described in connection with FIG. 2 may be hosted by one or more server racks 404. Such services may include job tracker 230, metadata manager 260, cleanup agent 232, job request balancer 238 and other services. Such services include services that do not handle frequent bulk data transfers. Finally, components described herein may communicate via connection 410, with other computing resources services 406 such as payload data cache 228, job tracker store 232, metadata manager job store 258 and the like as described in connection with FIG. 2 .", + "p0109": "FIG. 5 illustrates an example process 500 for storing data. Some or all of process 500 may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code executing collectively on one or more processors, by hardware or combinations thereof. The code may be stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory. One or more components of archival data storage system 206 as described in connection with FIG. 2 may perform process 500.", + "p0110": "Process 500 includes receiving 502 a data storage request to store archival data such as a document, a video or audio file or the like. Such a data storage request may include payload data and metadata such as size and digest of the payload data, user identification information , a logical data container identifier and the like. Process 500 may include receiving 502 multiple storage requests each including a portion of larger payload data. A storage request may include multiple data objects to be uploaded. Step 502 of process 500 is implemented by a service such as API request handler 218 of front end 208 as described in connection with FIG. 2 .", + "p0111": "Process 500 includes processing 504 the storage request upon receiving 502 the request. Such processing may include, for example, verifying the integrity of data received, authenticating the customer, authorizing requested access against access control policies, performing meter- and accounting-related activities and the like. Such processing may be performed by services of front end 208 such as described in connection with FIG. 2 . Such a request may be processed in connection with other requests, for example, in batch mode.", + "p0112": "Process 500 includes storing 506 the data associated with the storage request in a staging data store. Such staging data store may include a transient data store such as provided by payload data cache 228 as described in connection with FIG. 2 . Only payload data is stored in the staging store. Metadata related to the payload data may also be stored in the staging store. Data integrity is validated before being stored at a staging data store.", + "p0113": "Process 500 includes providing 508 a data object identifier associated with the data to be stored, for example, in a response to the storage request. As described above, a data object identifier may be used by subsequent requests to retrieve, delete or otherwise reference data stored. A data object identifier may encode storage location information that may be used to locate the stored data object, payload validation information such as size, digest, timestamp and the like that may be used to validate the integrity of the payload data, metadata validation information such as error-detection codes that may be used to validate the integrity of metadata such as the data object identifier itself and information encoded in the data object identifier and the like. A data object identifier may also encode information used to validate or authorize subsequent customer requests. For example, a data object identifier may encode the identifier of the logical data container that the data object is stored in. In a subsequent request to retrieve this data object, the logical data container identifier may be used to determine whether the requesting entity has access to the logical data container and hence the data objects contained therein. The data object identifier may encode information based on information supplied by a customer and/or information collected or calculated by the system performing process 500 . Generating a data object identifier may include encrypting some or all of the information described above using a cryptographic private key. The cryptographic private key may be periodically rotated. A data object identifier may be generated and/or provided at a different time than described above. For example, a data object identifier may be generated and/or provided after a storage job is created and/or completed.", + "p0114": "Providing 508 a data object identifier may include determining a storage location for the before the data is actually stored there. For example, such determination may be based at least in part on inventory information about existing data storage entities such as operational status , available storage space, data isolation requirement and the like. In an environment such as environment 200 illustrated by FIG. 2 , such determination may be implemented by a service such as storage node registrar 248 as described above in connection with FIG. 2 . Such determination may include allocating new storage space on one or more physical storage devices by a service such as storage allocator 256 as described in connection with FIG. 2 .", + "p0115": "A storage location identifier may be generated to represent the storage location determined above. Such a storage location identifier may include, for example, a volume reference object which comprises a volume identifier component and data object identifier component. The volume reference component may identify the volume the data is stored on and the data object identifier component may identify where in the volume the data is stored. In general, the storage location identifier may comprise components that identify various levels within a logical or physical data storage topology in which data is organized. The storage location identifier may point to where actual payload data is stored or a chain of reference to where the data is stored.", + "p0116": "A data object identifier encodes a digest of at least a portion of the data to be stored, such as the payload data. The digest may be based at least in part on a customer-provided digest. The digest may be calculated from scratch based on the payload data.", + "p0117": "Process 500 includes creating 510 a storage job for persisting data to a long-term data store and scheduling 512 the storage job for execution. In environment 200 as described in connection with FIG. 2 , steps 508, 510 and 512 may be implemented at least in part by components of control plane for direct I/O 210 and common control plane 212 as described above. Specifically, job tracker 230 creates a job record and stores the job record in job tracker store 232. As described above, job tracker 230 may perform batch processing to reduce the total number of transactions against job tracker store 232. Additionally, job tracker store 232 may be partitioned or otherwise optimized to facilitate parallel processing, cleanup operations and the like. A job record, as described above, may include j ob-related information such as a customer account identifier, job identifier, storage location identifier, reference to data stored in payload data cache 228, job status, job creation and/or expiration time and the like. A storage job may be created before a data object identifier is generated and/or provided. For example, a storage job identifier, instead of or in addition to a data object identifier, may be provided in response to a storage request at step 508 above.", + "p0118": "Scheduling 512 the storage job for execution includes performing job planning and optimization, such as queue-based load leveling or balancing, job partitioning and the like, as described in connection with common control plane 212 of FIG. 2 . For example, job request balancer 238 transfers job items from job request queue 236 to storage node manager job store 240 according to a scheduling algorithm so as to dampen peak to average load levels coming from control plane for I/O 210 and to deliver manageable workload to data plane 214. As another example, storage node manager job store 240 may be partitioned to facilitate parallel processing of the jobs by multiple workers such as storage node managers 244. As yet another example, storage node manager job store 240 may provide querying, sorting and other functionalities to facilitate batch processing and other job optimizations.", + "p0119": "Process 500 includes selecting 514 the storage job for execution, for example, by a storage node manager 244 from storage node manager job stored 240 as described in connection with FIG. 2 . The storage job may be selected 514 with other jobs for batch processing or otherwise selected as a result of job planning and optimization described above.", + "p0120": "Process 500 includes obtaining 516 data from a staging store, such as payload data cache 228 described above in connection with FIG. 2 . The integrity of the data may be checked, for example, by verifying the size, digest, an error-detection code and the like.", + "p0121": "Process 500 includes obtaining 518 one or more data encoding schemes such as an encryption scheme, a redundancy encoding scheme such as erasure encoding, redundant array of independent disks encoding schemes, replication, and the like. Such encoding schemes evolve to adapt to different requirements. For example, encryption keys may be rotated periodically and stretch factor of an erasure coding scheme may be adjusted over time to different hardware configurations, redundancy requirements and the like.", + "p0122": "Process 500 includes encoding 520 with the obtained encoding schemes. For example, data is encrypted and the encrypted data is erasure-encoded. Storage node managers 244 described in connection with FIG. 2 may be configured to perform the data encoding described herein. Application of such encoding schemes generates a plurality of encoded data components or shards, which may be stored across different storage entities such as storage devices, storage nodes, datacenters and the like to provide fault tolerance. Where data may comprise multiple parts , each part may be encoded and stored as described herein.", + "p0123": "Process 500 includes determining 522 the storage entities for such encoded data components. For example, in an environment 200 illustrated by FIG. 2 , a storage node manager 244 may determine the plurality of storage nodes 246 to store the encoded data components by querying a storage node registrar 248 using a volume identifier. Such a volume identifier may be part of a storage location identifier associated with the data to be stored. In response to the query with a given volume identifier, storage node registrar 248 returns a list of network locations of storage nodes 246 to store the encoded data components. As described in connection with FIG. 2 , storage node registrar 248 may determine such a list based on self-reported and dynamically provided and/or updated inventory information from storage nodes 246 themselves. Such determination is based on data isolation, fault tolerance, load balancing, power conservation, data locality and other considerations. Storage registrar 248 may cause new storage space to be allocated, for example, by invoking storage allocator 256 as described in connection with FIG. 2 .", + "p0124": "Process 500 includes causing 524 storage of the encoded data component at the determined storage entities. For example, in an environment 200 illustrated by FIG. 2 , a storage node manager 244 may request each of the storage nodes 246 determined above to store a data component at a given storage location. Each of the storage nodes 246, upon receiving the storage request from storage node manager 244 to store a data component, may cause the data component to be stored in a connected storage device. At least a portion of the data object identifier is stored with all or some of the data components in either encoded or unencoded form. For example, the data object identifier may be stored in the header of each data component and/or in a volume component index stored in a volume component. A storage node 246 may perform batch processing or other optimizations to process requests from storage node managers 244.", + "p0125": "A storage node 246 sends an acknowledgement to the requesting storage node manager 244 indicating whether data is stored successfully. A storage node 246 returns an error message, when for some reason, the request cannot be fulfilled. For example, if a storage node receives two requests to store to the same storage location, one or both requests may fail. A storage node 246 performs validation checks prior to storing the data and returns an error if the validation checks fail. For example, data integrity may be verified by checking an error-detection code or a digest. As another example, storage node 246 may verify, for example, based on a volume index, that the volume identified by a storage request is stored by the storage node and/or that the volume has sufficient space to store the data component.", + "p0126": "Data storage is considered successful when storage node manager 244 receives positive acknowledgement from at least a subset of requested storage nodes 246. A storage node manager 244 may wait until the receipt of a quorum of acknowledgement before removing the state necessary to retry the job. Such state information may include encoded data components for which an acknowledgement has not been received. To improve the throughput, a storage node manager 244 may remove the state necessary to retry the job before receiving a quorum of acknowledgement.", + "p0127": "Process 500 includes updating 526 metadata information including, for example, metadata maintained by data plane 214 , metadata maintained by control planes 210 and 212 , metadata maintained by metadata plane 216 and the like. Some of such metadata information may be updated via batch processing and/or on a periodic basis to reduce performance and cost impact. For example, in data plane 214, information maintained by storage node registrar store 250 may be updated to provide additional mapping of the volume identifier of the newly stored data and the storage nodes 246 on which the data components are stored, if such a mapping is not already there. For another example, volume index on storage devices may be updated to reflect newly added data components.", + "p0128": "In common control plane 212, job entries for completed jobs may be removed from storage node manager job store 240 and added to job result queue 242 as described in connection with FIG. 2 . In control plane for direct I/O 210, statuses of job records in job tracker store 232 may be updated, for example, by job tracker 230 which monitors the job result queue 242. A job that fails to complete may be retried for a number of times. For example, a new job may be created to store the data at a different location. As another example, an existing job record may be updated to facilitate retry of the same job.", + "p0129": "In metadata plane 216, metadata may be updated to reflect the newly stored data. For example, completed jobs may be pulled from job result queue 242 into metadata manager job store 258 and batch-processed by metadata manager 260 to generate an updated index such as stored in cold index store 262. For another example, customer information may be updated to reflect changes for metering and accounting purposes.", + "p0130": "Finally, once a storage job is completed successfully, job records, payload data and other data associated with a storage job may be removed, for example, by a cleanup agent 234 as described in connection with FIG. 2 . Such removal may be processed by batch processing, parallel processing or the like.", + "p0131": "FIG. 6 illustrates an example process 500 for retrieving data. One or more components of archival data storage system 206 as described in connection with FIG. 2 collectively perform process 600.", + "p0132": "Process 600 includes receiving 602 a data retrieval request to retrieve data such as stored by process 500, described above. Such a data retrieval request may include a data object identifier, such as provided by step 508 of process 500, described above, or any other information that may be used to identify the data to be retrieved.", + "p0133": "Process 600 includes processing 604 the data retrieval request upon receiving 602 the request. Such processing may include, for example, authenticating the customer, authorizing requested access against access control policies, performing meter and accounting related activities and the like. Such processing may be performed by services of front end 208 such as described in connection with FIG. 2 . Such request may be processed in connection with other requests, for example, in batch mode.", + "p0134": "Processing 604 the retrieval request may be based at least in part on the data object identifier that is included in the retrieval request. As described above, data object identifier may encode storage location information, payload validation information such as size, creation timestamp, payload digest and the like, metadata validation information, policy information and the like. Processing 604 the retrieval request includes decoding the information encoded in the data object identifier, for example, using a private cryptographic key and using at least some of the decoded information to validate the retrieval request. For example, policy information may include access control information that may be used to validate that the requesting entity of the retrieval request has the required permission to perform the requested access. As another example, metadata validation information may include an error-detection code such as a cyclic redundancy check that may be used to verify the integrity of data object identifier or a component of it.", + "p0135": "Process 600 includes creating 606 a data retrieval job corresponding to the data retrieval request and providing 608 a job identifier associated with the data retrieval job, for example, in a response to the data retrieval request. Creating 606 a data retrieval job is similar to creating a data storage job as described in connection with step 510 of process 500 illustrated in FIG. 5 . For example, a job tracker 230 may create a job record that includes at least some information encoded in the data object identifier and/or additional information such as a job expiration time and the like and store the job record in job tracker store 232. As described above, job tracker 230 may perform batch processing to reduce the total number of transactions against job tracker store 232. Additionally, job tracker store 232 may be partitioned or otherwise optimized to facilitate parallel processing, cleanup operations and the like.", + "p0136": "Process 600 includes scheduling 610 the data retrieval job created above. Scheduling 610 the data retrieval job for execution includes performing job planning and optimization such as described in connection with step 512 of process 500 of FIG. 5 . For example, the data retrieval job may be submitted into a job queue and scheduled for batch processing with other jobs based at least in part on costs, power management schedules and the like. For another example, the data retrieval job may be coalesced with other retrieval jobs based on data locality and the like.", + "p0137": "Process 600 includes selecting 612 the data retrieval job for execution, for example, by a storage node manager 244 from storage node manager job stored 240 as described in connection with FIG. 2 . The retrieval job may be selected 612 with other jobs for batch processing or otherwise selected as a result of job planning and optimization described above.", + "p0138": "Process 600 includes determining 614 the storage entities that store the encoded data components that are generated by a storage process such as process 500 described above. A storage node manager 244 may determine a plurality of storage nodes 246 to retrieve the encoded data components in a manner similar to that discussed in connection with step 522 of process 500, above. For example, such determination may be based on load balancing, power conservation, efficiency and other considerations.", + "p0139": "Process 600 includes determining 616 one or more data decoding schemes that may be used to decode retrieved data. Typically, such decoding schemes correspond to the encoding schemes applied to the original data when the original data is previously stored. For example, such decoding schemes may include decryption with a cryptographic key, erasure-decoding and the like.", + "p0140": "Process 600 includes causing 618 retrieval of at least some of the encoded data components from the storage entities determined in step 614 of process 600. For example, in an environment 200 illustrated by FIG. 2 , a storage node manager 244 responsible for the data retrieval job may request a subset of storage nodes 246 determined above to retrieve their corresponding data components. A minimum number of encoded data components is needed to reconstruct the original data where the number may be determined based at least in part on the data redundancy scheme used to encode the data . In such cases, the subset of storage nodes may be selected such that no less than the minimum number of encoded data components is retrieved.", + "p0141": "Each of the subset of storage nodes 246, upon receiving a request from storage node manager 244 to retrieve a data component, may validate the request, for example, by checking the integrity of a storage location identifier , verifying that the storage node indeed holds the requested data component and the like. Upon a successful validation, the storage node may locate the data component based at least in part on the storage location identifier. For example, as described above, the storage location identifier may include a volume reference object which comprises a volume identifier component and a data object identifier component where the volume reference component to identify the volume the data is stored and a data object identifier component may identify where in the volume the data is stored. The storage node reads the data component, for example, from a connected data storage device and sends the retrieved data component to the storage node manager that requested the retrieval. The data integrity is checked, for example, by verifying the data component identifier or a portion thereof is identical to that indicated by the data component identifier associated with the retrieval job. A storage node may perform batching or other job optimization in connection with retrieval of a data component.", + "p0142": "Process 600 includes decoding 620, at least the minimum number of the retrieved encoded data components with the one or more data decoding schemes determined at step 616 of process 600. For example, the retrieved data components may be erasure decoded and then decrypted. A data integrity check is performed on the reconstructed data, for example, using payload integrity validation information encoded in the data object identifier . In some cases, the retrieval job may fail due to a less-than-minimum number of retrieved data components, failure of data integrity check and the like. In such cases, the retrieval job may be retried in a fashion similar to that described in connection with FIG. 5 . The original data comprises multiple parts of data and each part is encoded and stored. During retrieval, the encoded data components for each part of the data may be retrieved and decoded to form the original part and the decoded parts may be combined to form the original data.", + "p0143": "Process 600 includes storing reconstructed data in a staging store such as payload data cache 228 described in connection with FIG. 2 . Data stored 622 in the staging store may be available for download by a customer for a period of time or indefinitely. Data integrity may be checked before the data is stored in the staging store.", + "p0144": "Process 600 includes providing 624 a notification of the completion of the retrieval job to the requestor of the retrieval request or another entity or entities otherwise configured to receive such a notification. Such notifications may be provided individually or in batches. The status of the retrieval job may be provided upon a polling request, for example, from a customer.", + "p0145": "FIG. 7 illustrates an example process 700 for deleting data. One or more components of archival data storage system 206 as described in connection with FIG. 2 collectively perform process 700.", + "p0146": "Process 700 includes receiving 702 a data deletion request to delete data such as stored by process 500, described above. Such a data retrieval request may include a data object identifier, such as provided by step 508 of process 500, described above, or any other information that may be used to identify the data to be deleted.", + "p0147": "Process 700 includes processing 704 the data deletion request upon receiving 702 the request. The processing 704 is similar to that for step 504 of process 500 and step 604 of process 600, described above. For example, the processing 704 is based at least in part on the data object identifier that is included in the data deletion request.", + "p0148": "Process 700 includes creating 706 a data retrieval job corresponding to the data deletion request. Such a retrieval job may be created similar to the creation of storage job described in connection with step 510 of process 500 and the creation of the retrieval job described in connection with step 606 of process 600.", + "p0149": "Process 700 includes providing 708 an acknowledgement that the data is deleted. Such acknowledgement may be provided in response to the data deletion request so as to provide a perception that the data deletion request is handled synchronously. A job identifier associated with the data deletion job may be provided similar to the providing of job identifiers for data retrieval requests.", + "p0150": "Process 700 includes scheduling 708 the data deletion job for execution. Scheduling 708 of data deletion jobs may be implemented similar to that described in connection with step 512 of process 500 and in connection with step 610 of process 600, described above. For example, data deletion jobs for closely-located data may be coalesced and/or batch processed. For another example, data deletion jobs may be assigned a lower priority than data retrieval jobs.", + "p0151": "Data stored may have an associated expiration time that is specified by a customer or set by default. A deletion job may be created 706 and schedule 710 automatically on or near the expiration time of the data. The expiration time may be further associated with a grace period during which data is still available or recoverable. A notification of the pending deletion may be provided before, on or after the expiration time.", + "p0152": "Process 700 includes selecting 712 the data deletion job for execution, for example, by a storage node manager 244 from storage node manager job stored 240 as described in connection with FIG. 2 . The deletion job may be selected 712 with other jobs for batch processing or otherwise selected as a result of job planning and optimization described above.", + "p0153": "Process 700 includes determining 714 the storage entities for data components that store the data components that are generated by a storage process such as process 500 described above. A storage node manager 244 may determine a plurality of storage nodes 246 to retrieve the encoded data components in a manner similar to that discussed in connection with step 614 of process 600 described above.", + "p0154": "Process 700 includes causing 716 the deletion of at least some of the data components. For example, in an environment 200 illustrated by FIG. 2 , a storage node manager 244 responsible for the data deletion job may identify a set of storage nodes that store the data components for the data to be deleted and requests at least a subset of those storage nodes to delete their respective data components. Each of the subset of storage node 246, upon receiving a request from storage node manager 244 to delete a data component, may validate the request, for example, by checking the integrity of a storage location identifier , verifying that the storage node indeed holds the requested data component and the like. Upon a successful validation, the storage node may delete the data component from a connected storage device and sends an acknowledgement to storage node manager 244 indicating whether the operation was successful. Multiple data deletion jobs may be executed in a batch such that data objects located close together may be deleted as a whole. Data deletion is considered successful when storage node manager 244 receives positive acknowledgement from at least a subset of storage nodes 246. The size of the subset may be configured to ensure that data cannot be reconstructed later on from undeleted data components. Failed or incomplete data deletion jobs may be retried in a manner similar to the retrying of data storage jobs and data retrieval jobs, described in connection with process 500 and process 600, respectively.", + "p0155": "Process 700 includes updating 718 metadata information such as that described in connection with step 526 of process 500. For example, storage nodes executing the deletion operation may update storage information including index, free space information and the like. Storage nodes may provide updates to storage node registrar or storage node registrar store. Some of such metadata information may be updated via batch processing and/or on a periodic basis to reduce performance and cost impact.", + "p0156": "FIG. 8 depicts an illustrative flow 800 in which techniques for the validation of data integrity may be implemented. These techniques are described in more detail below in connection with FIGS. 9-13 . Returning to FIG. 8 , in illustrative flow 800, operations may be performed by one or more processors of an archival data storage service and/or instructions that perform the operations may be stored in one or more memories of the archival data storage service. As desired, the flow 800 may begin at 802, where the archival data storage service may receive one or more parts of a data payload 804. In some examples, the data payload 804 may include any number of parts; however, in this example two parts are shown, Part 1 and Part 2. Each of Part 1 and Part 2 may include data 806 and 808, respectively. In some cases, the size of Part 1 and Part 2 may be selected by the uploader and/or may be the same. However, in some examples, the last part of a data payload 804 may be a different size from all the other consistently sized parts . At 810, the flow 800 may generate sub-parts of the parts of the payload 804. In some examples, the size of the sub-parts may be predefined by the archival data storage service .", + "p0157": "In some examples, the flow 800 may calculate a digest for each sub-part at 812. The respective digests may be stored as nodes 814 of a data structure such as, but not limited to, the data structure 816 generated at 818. By way of example only, the data structure 816 may include one or more sub-part digests and/or one or more part digests . Additionally, at 824, the flow 800 may determine a root digest 826 for the root of the data structure 816. In some examples, the root digest 826 may be determined or generated based at least in part on concatenating part digests and calculating a digest for the concatenated digests. The flow 800 may end at 828, where the archival data storage service may verify that the received payload 804 matches a stored payload 830. The stored payload may, in some examples, contain data 832 determined based at least in part on combining each of the parts 806, 808 and/or sub-parts, when received. In some examples, verifying the data payload may be based at least in part on comparing the root digest 826 against a second root digest received from the uploader.", + "p0158": "FIG. 9 depicts an illustrative data structure 900 in which additional techniques for the validation of data integrity may be implemented. Illustrative data structure 900 is but one of many different types of data structures that may be utilized to implement the techniques described herein. By way of example only, a user or client entity may wish to upload a data payload 902 to the archival data storage service. The archival data storage service may then be configured to receive the data payload 902 and allow the user to verify, at some point , that the data stored in the archival data storage service is, in fact, the same as the data payload 902 that was uploaded without requesting any size partitioning information from the user. In other words, the archival data storage service may provide a data object identifier that the user may return in order to retrieve stored data; however, the user may not need to store any information other than the data object identifier.", + "p0159": "In some examples, in order to accept data from the user, the archival data storage service may request that the user provide a tree digest like the data structure 900 of FIG. 9 . Providing the data structure 900 may be performed in multiple ways. For example, all of the data illustrated in in the data structure 900 may be provided. As an alternative, where the data structure 900 is constructible solely from the data for the leaf nodes, data for the leaf nodes may be provided without providing information for other, higher-level nodes. Additionally, the archival data storage service may provide instructions in the form of an algorithm, API and/or SDK for generating the data structure 900. In some instances, limitations on the size of upload chunks and their respective offsets may be imposed. For example, the chunks or parts of the data payload 902 may be limited to powers of two of 1MB. Additionally, in some examples, the determined size of each chunk may not be changed within a particular upload. Further, for each part received, the archival data storage service may calculate its own digest, based at least in part on the same algorithm used by the user, and provide the digest for each part. Upon completion of the storage job, the archival data storage service may provide the top-level digest value in the form of a data object identifier. Retrieval of the data may, in some examples, be implemented in a similar fashion, with restrictions on chunk sizes and offsets limited to powers of two by 1MB, messages prepended with the digest of the data that is in the message and the top-level digest available upon completion of the job. However, based at least in part on this implementation, the data payload 902 should be able to be verified or validated independent of the chunk size selected by the user. A digest may be calculated by applying a cryptographic hash function such as those associated with SHA-1, SHA-2, MD5, MD6 and the like, a checksum or error-detection code such as cyclic redundancy check and the like to at least a portion of the payload data.", + "p0160": "The data structure 900 of FIG. 9 may illustrate an appropriate digest tree for a data payload 902 where the user has chosen to upload the data payload in a single part. As such, there is no part size for the user to select in this example. However, the resulting root digest 906 should be calculable using the techniques described herein even if the user had selected to upload the data payload 902 in multiple parts, and even if the user had selected a part size unknown to the archival data storage service and/or not recorded by the user. In this example, for the sake of simplicity, it will be assumed that the data payload 902 is 7MBs in size. As such, and since the user has requested to upload the entire payload 902 in one part, the data payload 902 may be partitioned into seven 1MB chunks, Sub 1 - Sub 7. In some examples, however, if the size of the payload 902 were not divisible by 1MB, the last chunk, Sub 7, may be smaller than 1MB. The archival data storage service, based at least in part on the hash tree algorithm, generates a hash value for each 1MB chunk . Each of these hash values is represented at the lowest child node level 908 of the data structure 900. In order to generate the nodes of the second child node level 910, the archival data storage service may concatenate each pair of second-level node children and run the hash function on the concatenated data. In other words, the lowest level 908 of the data structure may include a digest of payload data, while parent nodes may include digests of digests. Moving up the data structure, the described operations may be repeated until a root digest 906 is generated.", + "p0161": "As described, in some cases, the archival data storage service may provide intermediary root digests for individual parts of the payload 902. However, in this example, since the payload was not broken into parts, the archival data storage service may only provide the root digest 906 to the user. In some cases, though, the archival data storage service may also provide each 1MB digest generated. As such, either the user or the archival data storage service should be able to verify that the data was uploaded correctly based at least in part on comparing each other's generated root digest 906.", + "p0162": "FIG. 10 depicts another illustrative data structure 1000 in which additional techniques for the validation of data integrity may be implemented. As noted with reference to FIG. 9 , the illustrative data structure 1000 is but one of many different types of data structures that may be utilized to implement the techniques described herein. By way of example only, a user or client entity may wish to upload a data payload 1002 to the archival data storage service. The archival data storage service may then be configured to receive the data payload 1002 and allow the user to verify that the data stored in the archival data storage service is, in fact, the same as the data payload 1002 that was uploaded. This validation may be done without requesting any size partitioning information from the user. In other words, the archival data storage service may provide a data object identifier that the user may return in order to retrieve stored data; however, the user may not need to store any information other than the data object identifier in order to request and/or validate the stored data.", + "p0163": "In generating the data structure 1000, the user or the archival data storage service may once again break the data into sub-parts; however, in this example, each part Part 1 or Part 2 may be broken up separately . Again, a digest for each sub-part may be generated and included in the data structure at the child level 1004 and digests of concatenated digests may be generated and included in the data structure at a first parent level 1006. In this example, however, since the payload 1002 has been broken into two parts, a top-level digest may be generated for each part. As such, Part 1 digest 1008 and Part 2 digest 1010 may be generated and included in the data structure 1000. Additionally, as the payload 1002 is uploaded, each of the sub-part digests and the part digests may be included in the upload. Further, a root digest 1012 may be generated in the same fashion that the other parent nodes are generated. That is, based at least in part on concatenating the children digests, and running the hash function on the concatenated information. In this example, this process would entail concatenating Part 1 digest 1008 and Part 2 digest 1010 to generate a part-level digest. The archival data storage service may then run the hash function on the part-level digest to generate the root digest 1006. In some examples, the root digest may be received at the beginning of upload and once the upload is completed. Additionally, the archival data storage service may generate its own version of the data structure 1000 and/or the root digest 1006 in order to validate the integrity of the data. Further, in some examples, the root digest 1006 generated by the archival data storage service may be provided to the user as part of a data object identifier that the user may utilize to make read, delete or index viewing requests.", + "p0164": "In some examples, the archival data storage service may assume that data corruptions can occur anywhere in the system and/or may be caused by hardware bugs, bit flips and/or due to bugs in software code implemented by the archival data storage service or the user. For at least this reason, the archival data storage service may review all, or a subset, of the data paths and operations to ensure that data integrity is provided throughout the system and that corrupt data is detected. In some cases, this may apply to the data payload and to the metadata. As such, data integrity validation may be performed on the data object identifiers as well to ensure that requests to delete data are not pointing at the wrong data.", + "p0165": "In some aspects, the archival data storage service 206 may be configured to expect that a selected or otherwise determined digest function may be acceptable for the validation of data integrity. In some examples, the digest function may not be used for some cases related to data transformation. Otherwise, it may be selected and/or provided for use with validating some, al, or portions of the data and/or metadata of the archival data storage service. Additionally, as noted, in some examples, the initiator may pre-calculate the digest of the data payload 1002 before transmission and then later may supply the digest again with the data to the archival data storage service. The archival data storage service may then recalculate the digest , compare with the digest received from the initiator and/or acknowledge that the integrity of the data was validated by providing the archival data storage service-generated digest to the user. Additionally, each data subdivision and/or aggregation may be re-validated by calculating the independent digest on the split of the aggregate data and comparing the digest or even by performing a bit by bit comparison. In other words, given any data payload, of any size, calculations may be performed to generate any number or type of the split or aggregated digests and, thus, validate the data and/or the parts.", + "p0166": "Additionally, in some aspects, data transformation such as, but not limited to, erasure coding or encryption can be re-validated by performing the reverse transformation. The results of the reverse transformation may then be cross checked by comparing the digest and/or by bit by bit comparison. As such, the transformed data may include two digests. One of the two may testify to the integrity of the transformed data and the other may testify to the integrity of the original data. In some examples, referential items such as, but not limited to, the data object identifier that may reference the content may include the digest of the data being referenced. Additionally, the archival data storage service may also include information about the parent node that is being referenced. In some cases, messages from the control plane that are persisted in the storage node registrar store 250, the data object identifier and/or other data structures may include digests that self-validate. These digests may be produced after the structures are created and/or verified upon retrieval or before the action. In some examples, this prevent things such as bugs in the code, memory corruption or bit rot from flipping data object retrieve commands into delete commands. Further, on the return path, the archival data storage service may be configured to re-validate that the data that is being returned to the customer is matched against the request and/or that no substitution during execution happens due to a bug in the code or other the like.", + "p0167": "FIGS. 11-13 illustrate example flow diagrams showing respective processes 1100-1300 for providing validation of data integrity. These processes are illustrated as logical flow diagrams, each operation of which represents a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.", + "p0168": "Additionally, some, any, or all of the processes may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code executing collectively on one or more processors, by hardware, or combinations thereof. As noted above, the code may be stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory.", + "p0169": "In some aspects, the API request handler 218, the payload data cache 228, the storage node manager 244 and/or the storage nodes 246 of the one or more archival data storage service 206 shown in FIG. 2 may perform the process 1100 of FIG. 11 . The process 1100 may begin by providing a function call for requesting data storage at 1102. The function call may be part of an API or an SDK for interacting with and/or interfacing with the archival data storage service. At 1104, the process 1100 may include receiving a plurality of portions of a data payload from a remote computing device . In some cases, the size of each portion may be consistent. In other cases, the size of each portion may be consistent except that the last portion may be different. Additionally, the size may be selected or otherwise determined by the user. At 1106, the process 1100 may include receiving an indication of the size of each portion. In some instances, the actions performed at 1104 and 1106 may be performed together as a single action. However, some restrictions may apply regarding portion size. Such as, in some examples, the portions may be limited to a consistent size ; however, the last portion may be a remainder of the data payload . For example, the size selection may be limited to 1MB or an integer multiple of 1MB. In other examples, the size may be limited to 1MB or a power of two of 1MB.", + "p0170": "The process 1100 may also include generating one or more sub-portions of a predefined size for at least some of the portions of the payload at 1108. As noted above, regarding the portion size, while sub-portion size may be predefined and constant, the last sub-portion may be a different size. The predefined size may be 1MB or any other size. At 1110, the process 1100 may include calculating one or more digests or hash values based at least in part on the sub-portions. The digests may be calculated based at least in part on a published and/or otherwise provided algorithm. In some examples, at 1112, the process 1100 may include generating a root node of a data structure. The data structure may be based at least in part on the sub-portion digests and/or aggregated digests as described above. At 1114, the process 1100 may include determining a top-level digest of the data structure. The top-level digest may be based at least in part on the root node of the data structure and/or on a parent node associated with one of the portions of data. At 1116, the process 1100 may include providing instructions configured to enable the remote computing device to generate the data structure. In this way, the user may generate the data structure along with the archival data storage service. The process 1100 may then include receiving a top-level digest generated by the remote computing device at 1118. The process 1100 may end at 1120 by verifying that the stored data payload matches a received data payload. In other words, the process 1100 may validate or verify the integrity of the data.", + "p0171": "FIG. 12 illustrates a flow diagram showing process 1200 for validating the integrity of data in accordance an embodiment of the invention. The API request handler 218, the payload data cache 228, the storage node manager 244 and/or the storage nodes 246 of the one or more archival data storage service 206 shown in FIG. 2 may perform the process 1200 of FIG. 12 . The process 1200 begins by receiving one or more parts of a data payload at 1202. As noted above, the parts may be any size. However, in some examples, the part size may be limited to 1MB or multiples of 1MB. In this way, a data structure may be composed independent of the chosen size. At 1204, the process 1200 includes generating a sub-part for the one or more parts. Again, these sub-parts may be any size or may be limited to 1MB or other size limitation such as, but not limited to, 2MB, 10MB, etc. the process 1200 includes calculating a value based on the sub-part at 1206. The value is a digest. The process 1200 includes generating a root node of a data structure at 1208. At 1210, the process 1200 includes determining a top-level value of the data structure based at least in part on traversing the data structure to the root node.", + "p0172": "The process 1200 also includes storing the data payload at 1212. The payload may be stored based at least in part on combining each of the one or more parts received at 1202. As such, in some cases, the archival data storage service 206 may not be able to store the payload at 1212 until the data transmission of all the parts is complete. At 1214, the process 1200 includes validating that the stored data payload matches the received data payload. This may be performed by comparing the received top-level value with a calculated top-level value. At 1216, the process 1200 may include providing a data object identifier including the top-level value. The identifier may later be utilized by the user to retrieve and/or delete the stored data payload. In some examples, the process 1200 may include receiving a request for the stored payload at 1218. The stored payload may be provided back to the user in a similar fashion . However, in some cases, the process 1200 may end at 1220 by verifying that the stored data payload has not changed prior to providing the payload.", + "p0173": "FIG. 13 illustrates another example flow diagram showing process 1300 for validating the integrity of data. In some aspects, the API request handler 218, the payload data cache 228, the storage node manager 244 and/or the storage nodes 246 of the one or more archival data storage service 206 shown in FIG. 2 may perform the process 1300 of FIG. 13 . The process 1300 may begin by providing instructions for making method calls to perform operations on data at 1302. In some examples, these method calls may be exposed via one or more APIs or provided in one or more SDKs. At 1304, the process 1300 may include performing a first operation, using a verification algorithm, based on a first partitioning of a data object into first partitions. The first partitions, in some examples, may include 1MB or other consistent sized chunks that may be utilized to generate a data structure such as, but not limited to, a hash tree or other binary tree of digests. In some examples, the first operation may include receiving the data from a user over a network. At 1306, the process 1300 may include verifying the data object to generate a first verification value based on the first partitions. The process 1300 may also include performing a second operation on a data object, utilizing the same verification algorithm, based at least in part on a second partitioning of the data object into second partitions at 1308. The second partitions may be a different size from the first partitions. Based at least in part on the second partitions, the process 1300 may include verifying the data object to generate a second value at 1310. Here, the second operation may also include transmitting data to the archival data storage service. The second verification value, like the first may include, but is not limited to, a digest for a partition, a digest for digests formed by aggregating partition digests and/or a top-level digest of a data structure. At 1312, the process 1300 may end by determining whether the second verification value equals the first verification value. This may be determined based at least in part on comparing the two values. In some examples, if the verification algorithm is properly performed, and the data has maintained its integrity, the two values are expected to be equal. That is, independent of the size of the two sets of partitions , the verification values should be equal.", + "p0174": "FIG. 14 depicts an illustrative architecture 1400 in which techniques for storage drive rotation scheduling may be implemented. In illustrative architecture 1400, an archival data storage system 1402 may provide one or more archival services including, but not limited to, archival data storage, batch job processing, data retrieval, data management, storage device management and/or rotation scheduling. In some aspects, the archival data storage system 1402 may determine and/or store a rotation schedule 1404. As noted above, the rotation schedule 1404 may indicate which drives are to be activated and when. Additionally, the rotation schedule 1404 may be determined based at least in part on several factors and/or may be provided by a data storage node manager and/or a data storage node. In some examples, the rotation schedule 1404 may be provided to a controller 1406. In some examples, the controller 1406 may include the one or more storage nodes noted above. Alternatively, or in addition, the controller 1406 may include, but is not limited to, a dedicated memory controller, a Small Computer System Interface controller, an Integrated Drive Electronics controller, a Fibre Channel controller, a Serial Attached SCSI controller or the like. Additionally, the controller 1406 could be an abstraction of such controllers, or some \"supercontroller\" that communicates with such drive controllers via the protocols listed. By way of example only, the controller 1406 may be configured to control one or more hard drive ) groups of a rack 1408 or other configuration based at least in part on one or more instructions 1410. For example, instruction 1412 may correspond to an action to be performed on HDD Group 1, instruction 1414 may correspond to HDD Group 2 and so on. In this way, the controller 1406 may be configured to perform different instructions on each HDD Group based at least in part on the schedule 1404.", + "p0175": "In one non-limiting example, the controller 1406 may follow the schedule 1404 to activate HDD Group 4 based at least in part on instruction 1418, labeled \"ON.\" That is, instruction 1418 may correspond to HDD Group 4 and may indicate that HDD Group 4 should be active for a predetermined amount of time. As noted above, the amount of time each group may be active may be based at least in part on several factors including, but not limited to, a portion of an hour based at least in part on a division of groups and/or some on-demand requests of the storage node. After the predefined time period has expired, the controller 1406 may move the \"ON\" instruction 1418 down the chain of instructions 1410 such that HDD Group 5 may be active next. As such, instruction 1420 is labeled \"Power Up,\" to indicate that HDD Group 5 may be scheduled to begin powering up at some point in the future. Similarly, instruction 1416 is labeled \"Power Down,\" to indicate that HDD Group 3 may still be powering down after being active. In at least one example, after a predefined time , the controller 1406 may shift the instructions down one such that instruction 1418 may be labeled \"ON,\" and HDD Group 5 may become activated. Other scenarios and/or examples may be possible for scheduling rotation of the drives. For example, each HDD group may include only a single storage drive such that each HDD Group may include one or more drives and is not limited to a scenario where each group includes multiple drives. Additionally, as noted above, additional instructions may be processed in order to avoid power consumption spikes based at least in part on activating multiple drives at the same time. For example, the drives of a group that are scheduled to rotate during a certain period may be scheduled to activate one at a time or in sub-groups as desired. Additionally, the HDD Groups may not be listed and/or may not be in any particular order, as shown. For example, the controller 106 may instead keep track in a table, chart, index, etc., which drives or Groups have been activated during a particular period and randomly cycle through activation of drives that have yet to be activated during the period. Thus, the controller 1406 may be free to determine which drives should be activated and when without being limited by the grouping or list shown in FIG. 14 .", + "p0176": "FIG. 15 depicts an illustrative architecture 1500 in which additional techniques for storage drive rotation scheduling may be implemented. In illustrative architecture 1500, an archival data storage system 1502 may provide one or more archival services including, but not limited to, archival data storage, batch job processing, data retrieval, data management, storage device management and/or rotation scheduling. As such, in some aspects, the archival data storage system 1502 may determine and/or store a rotation schedule 1504. As noted above, the rotation schedule 1504 may indicate which drives of a storage service are to be activated and/or during what time periods. Additionally, the rotation schedule 1504 may be determined based at least in part on several factors and/or may be provided by the data storage node manager 224 and/or the data storage node, as described above.", + "p0177": "In some aspects, a controller 1506 may be configured to control one or more hard drives 1508 , ..., 1508 . By way of example only, the hard drives 1508 are illustrated in an array or ordered list; however, any ordering and/or other organization may be envisioned for determining which particular hard drives 1508 or sets of hard drives come before or after other particular hard drives 1508 . For example, the array may be utilized by the controller for pointing to a next hard drive to activate and not for ordering the hard drives 1508. However, in other examples, the array may be utilized to dynamically order the hard drives .", + "p0178": "In some examples, a window 1510 may be envisioned for activating the hard drives 1508. Based at least in part on a predefined period of time, the window 1510 may move along the array to become windows 1512, 1514 and 1516, for example, at each period. In other words, the window 1510 may represent the hard drives 1508 that are active or are to be activated by the controller 1506 during a first period. That is, by way of example and not limitation, during the first period, the hard drives 1508 -1508 may be active . In the example shown in FIG. 15 , at the end of the first period, the window 1510 may move to become the window 1512, where hard drives 1508 -1508 become active. In some examples, when the transition occurs , the hard drive 1508 may be powered down prior to the controller 1506 activating the hard drive 1508 . In this way, the power draw associated with operating the array of hard drives 1508 may not exceed a certain level.", + "p0179": "Additionally, in some aspects, the windows 1510, 1512, 1514, 1516 may be defined by a length 1518. The length may be determined based at least in part on demand, a percentage of the total hard drives 1508, a predetermined number of the total hard drives 1508, some predetermined level of power consumption, temperature, cost, etc. or the like. In addition, the window may be configured to minimize power consumption of the hard drives 1508, minimize operational costs associated with the hard drives 1508, and/or maximize the operational lifetime of the hard drives 1508 . As such, the length 1518 may be constant throughout the rotation schedule 1504 or it may be dynamic. In some examples, once the window 1510 reaches the end of the array of hard drives 1508, the window 1510 may cycle back to the beginning. Pictorially, this may be represented as a circular array, where the hard drive 1508 would follow the hard drive 1508 . Further, in some examples, an array as shown in FIG. 15 may not be used at all. Instead, any other form or type of data structure may be utilized to organize the hard drives 1508 in such a way that the rotation schedule 1504 may determine and/or control which hard drives 1508 are to be activated at what times. Additionally, each of the hard drives 1508 may actually be a group of one or more hard drives instead of a single drive. In this scenario, the controller 1506 may be further configured to stagger the activation of each single hard drive within the group of hard drives such as to limit power spikes or other potential power consumption issues.", + "p0180": "In some examples, the hard drives 1508 may be arranged in a logical order, such as shown with the array 1520. The hard drives 1508 may also be arranged in a server or on a rack in a physical order that may be different from the logical order, such as shown with illustrative physical rack 1522. That is, the hard drives 1508 may actually be located on the physical rack 1522 in the order shown as opposed to the order shown in the logical array 1520. Thus, as the schedule 1504 moves the window 1510 along the logical order 1520, the hard drives 1508 may be activated or otherwise powered up based at least in part on their logical order and/or their physical location. Additionally, a matrix 1524 may indicate which hard drives 1508 are activated at each unit of time, according to the schedule 1504. In some examples, each row or column of the matrix 1524 may represent physical racks in a server or cabinet such that adjacent rows may represent adjacent racks. Additionally, in some examples, one or more constraints may be provided and/or implemented such that an even or other distribution of hard drives 1508 on the matrix 1524 may be ensured. For example, in order to evenly distribute the enabled hard drives 1508, the conditions or constraints may indicate that hard drives 1508 of a particular temperate, age, location, amount of vibration or the like may not be activated that are within a particular distance of other hard drives 1508 of the matrix 1524. For example, the schedule 1504 may indicate that a particular hard drive 1508 is to be activated; however, the condition may instruct the archival data storage service 1502 to modify the indication or provide an instruction to activate a different hard drive 1508 instead.", + "p0181": "As described with reference to FIG. 15 , in some aspects, the total number of hard drives 1508 may be divided into a predefined number of groups. Then, these groups may be rotated in and out of operation based at least in part on the rotation schedule 1504. Such an approach may allow the storage node registrars 248 to determine when and where to place new write operations or when it may be possible to execute read or delete operations. In some examples, the data storage node managers 224 may be isolated from this knowledge and may follow directions from the storage node registrars 248. However, in other examples, the rotation schedule 804 may be determined on-demand. For example, based at least in part on read requests from one or more data storage node managers 224, a storage node registrar 248 may select a number of the hard drives 808 that should come in to a rotation, and then send messages to the data storage nodes. Alternatively, the data storage node manager 224 may directly contact the data storage nodes. In some examples, particular constraints may influence the rotation schedule 1504 including, but not limited to, a goal not to starve, for too long, a request to a drive which has only one read request outstanding.", + "p0182": "FIGS. 16-18 illustrate example flow diagrams showing respective processes 1600-1800 for providing data storage power management. These processes are illustrated as logical flow diagrams, each operation of which represents a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.", + "p0183": "Additionally, some, any or all of the processes may be performed under the control of one or more computer systems configured with executable instructions and may be implemented as code executing collectively on one or more processors, by hardware, or combinations thereof. As noted above, the code may be stored on a computer-readable storage medium, for example, in the form of a computer program comprising a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory.", + "p0184": "In some aspects, the storage node manager 244, the storage nodes 246, the storage node registrar store 250 and/or other data storage devices or collections of the archival data storage service 206 shown in FIG. 2 may perform the process 1600 of FIG. 16 . The process 1600 may begin with classification of one or more data storage devices of the archival data storage service 206 into groups at 1602. In some examples, each group may be a subset or part of a subset of the data storage devices. Additionally, classifying may involve determining an amount or percentage of storage devices to be grouped together based at least in part on characteristics of the storage devices such as, but not limited to, location, power consumption, age, speed, cost, etc. At 1604, the process may include modifying a power state of the groups according to a schedule. The schedule may be configured to ensure that, on average, an amount of the data storage devices are in a power down state. The power down state may be a low power state, a sleep mode, a powering down state, off or the like. Additionally, the amount of devices to be in a power down state may be based at least in part on a desired level of power consumption and/or cost associated with operating the archival data storage service 206. Further, in some examples, the schedule may be received or otherwise based at least in part on the storage node registrar store 250, the storage node registrar 248 and/or the data storage node.", + "p0185": "In some examples, the process 1600 may also include obtaining a batch of data processing requests at 1606. The data processing requests may be part of a batch or finite grouping of requests or they may be part of a queue for more synchronous operation. Additionally, the batch of requests may include read requests, index list requests, write requests, delete requests or the like. At 1608, the process 1600 may include processing the batch requests. In some examples, processing the requests may include identifying a storage device in a power down state that, when powered on, would be usable to service the request at 1610. Additionally, processing the requests may also include servicing the request when the group containing the identified storage device is modified according to the schedule to be in the power up state at 1612. That is, once the identified storage device is powered up or otherwise enabled, the archival data storage service 206 may service the request . Additionally, in some cases, the actions of 1608, 1610 and/or 1612 may be processed as a single action, two actions or three separate actions. In some examples, the process 1600 may end at 1612. However, in other example, the process 1600 may end with powering down a group of storage devices in the power up state. Alternatively, or in addition, the process 1600 may include powering down the group of storage devices in the power up state prior to powering up the group including the identified storage device and/or prior to servicing the request at 1614.", + "p0186": "FIG. 17 illustrates another example flow diagram showing process 1700 for data storage power management. In some aspects, the storage node manager 244, the storage nodes 246, the storage node registrar store 250 and/or other data storage devices or collections of the archival data storage service 206 shown in FIG. 2 may perform the process 1700 of FIG. 17 . The process 1700 may begin with reception of a rotation schedule, agenda or other type of timetable, at 1702, from the data storage node 246, the storage node registrar 248 or the storage node registrar store 250. As noted, the schedule may be performed by the data storage node 246. As noted above, the schedule may include instructions for which storage devices are to be enabled or otherwise powered on and when. In some examples, the schedule may also include instructions for device controllers that may be configured to managing the power consumption of each storage device by slowing, stopping or speeding up disks and/or by disabling the storage device by removing an electrical signal. At 1704, the process 1700 may include managing power of one or more storage devices based at least in part on the schedule.", + "p0187": "The process 1700 may also include identifying one or more data processing requests of a batch of requests associated with the storage devices at 1706. As noted above, the requests may be received in a single batch. Additionally, the requests may be processed in a batch at 1708. However, in some examples, the requests may be received in a batch and processed sequentially, yet no new requests may be received until the archival data storage service 206 has completed some or all of the requests of the batch. At 1710, the process 1700 may include identifying a data storage device in a low power mode that is able to service the request when powered up. Further, the process 1700 may end with service of the request once the identified storage device is powered up based on the schedule. In this way, the schedule may control the power management of the storage devices, while at the same time, the batch of requests may influence the schedule. In this way, the data storage power management described herein may be considered on-demand. Additionally, in some aspects, the processing of the data processing requests at 1708 may include the actions of 1710 and/or 1712. As such, they may all be performed in a single action.", + "p0188": "FIG. 18 illustrates another example flow diagram showing process 1800 for validating the integrity of data. In some aspects, the storage node manager 244, the storage nodes 246, the storage node registrar store 250 and/or other data storage devices or collections of the archival data storage service 206 shown in FIG. 2 may perform the process 1800 of FIG. 18 . The process 1800 may begin, at 1802, with reception of a global time signal configured to enable management of the power consumption of one or more storage devices to be synchronized with access of the storage devices. That is the global time signal may be utilized by the archival data storage service 206 to accurately time when to manage power of each storage device. Additionally, the archival data storage service 206 may utilize the global time signal to determine when to access the data storage devices. As such, the two actions may, in some examples, by synchronized such that a storage device is accessed after its power consumption has been managed such that it is on, powered up or otherwise accessible without significant delay. At 1804, the process 1800 may include obtaining a schedule associated with enabling a first storage device at a different time than a second storage device, where each of the two storage device are part of a set of storage devices. In other words, the schedule may include information that indicates which storage devices of the set should be powered up and when including, but not limited to, when the first and second devices should be powered up. The process 1800 may also include identifying a request of a batch of requests for accessing the set of storage devices at 1806. As noted above, the requests may include read requests, write requests, delete requests, index list requests, or the like. At 1808, the process 1800 may include determining which storage device to enable based at least in part on the schedule. The determination may include selecting the first storage device, the second storage device or any other storage device of the set of storage devices being controlled or managed by the archival data storage service 206. Further, at 1810, the process 1800 may end with the management of power consumption of the determined storage device based at least in part on the determination from 1808.", + "p0189": "Illustrative methods and systems for validating the integrity of data are described above. Some or all of these systems and methods may, but need not, be implemented at least partially by architectures such as those shown above.", + "p0190": "FIG. 19 illustrates aspects of an example environment 1900 for implementing aspects in accordance with various embodiments. As will be appreciated, although a Web-based environment is used for purposes of explanation, different environments may be used, as appropriate, to implement various embodiments. The environment includes an electronic client device 1902, which can include any appropriate device operable to send and receive requests, messages or information over an appropriate network 1904 and convey information back to a user of the device. Examples of such client devices include personal computers, cell phones, handheld messaging devices, laptop computers, set-top boxes, personal data assistants, electronic book readers and the like. The network can include any appropriate network, including an intranet, the Internet, a cellular network, a local area network or any other such network or combination thereof. Components used for such a system can depend at least in part upon the type of network and/or environment selected. Protocols and components for communicating via such a network are well known and will not be discussed herein in detail. Communication over the network can be enabled by wired or wireless connections and combinations thereof. In this example, the network includes the Internet, as the environment includes a Web server 1906 for receiving requests and serving content in response thereto, although for other networks an alternative device serving a similar purpose could be used as would be apparent to one of ordinary skill in the art.", + "p0191": "The illustrative environment includes at least one application server 1908 and a data store 1910. It should be understood that there can be several application servers, layers, or other elements, processes or components, which may be chained or otherwise configured, which can interact to perform tasks such as obtaining data from an appropriate data store. As used herein the term \"data store\" refers to any device or combination of devices capable of storing, accessing and retrieving data, which may include any combination and number of data servers, databases, data storage devices and data storage media, in any standard, distributed or clustered environment. The application server can include any appropriate hardware and software for integrating with the data store as needed to execute aspects of one or more applications for the client device, handling a majority of the data access and business logic for an application. The application server provides access control services in cooperation with the data store, and is able to generate content such as text, graphics, audio and/or video to be transferred to the user, which may be served to the user by the Web server in the form of HTML, XML or another appropriate structured language in this example. The handling of all requests and responses, as well as the delivery of content between the client device 1902 and the application server 1908, can be handled by the Web server. It should be understood that the Web and application servers are not required and are merely example components, as structured code discussed herein can be executed on any appropriate device or host machine as discussed elsewhere herein.", + "p0192": "The data store 1910 can include several separate data tables, databases or other data storage mechanisms and media for storing data relating to a particular aspect. For example, the data store illustrated includes mechanisms for storing production data 1912 and user information 1916, which can be used to serve content for the production side. The data store also is shown to include a mechanism for storing log data 1914, which can be used for reporting, analysis or other such purposes. It should be understood that there can be many other aspects that may need to be stored in the data store, such as for page image information and to access right information, which can be stored in any of the above listed mechanisms as appropriate or in additional mechanisms in the data store 1910. The data store 1910 is operable, through logic associated therewith, to receive instructions from the application server 1908 and obtain, update or otherwise process data in response thereto. In one example, a user might submit a search request for a certain type of item. In this case, the data store might access the user information to verify the identity of the user, and can access the catalog detail information to obtain information about items of that type. The information then can be returned to the user, such as in a results listing on a Web page that the user is able to view via a browser on the user device 1902. Information for a particular item of interest can be viewed in a dedicated page or window of the browser.", + "p0193": "Each server typically will include an operating system that provides executable program instructions for the general administration and operation of that server, and typically will include a computer-readable storage medium storing instructions that, when executed by a processor of the server, allow the server to perform its intended functions. Suitable implementations for the operating system and general functionality of the servers are known or commercially available, and are readily implemented by persons having ordinary skill in the art, particularly in light of the disclosure herein.", + "p0194": "The environment in one embodiment is a distributed computing environment utilizing several computer systems and components that are interconnected via communication links, using one or more computer networks or direct connections. However, it will be appreciated by those of ordinary skill in the art that such a system could operate equally well in a system having fewer or a greater number of components than are illustrated in FIG. 19 . Thus, the depiction of the system 1900 in FIG. 19 should be taken as being illustrative in nature, and not limiting to the scope of the disclosure.", + "p0195": "The various embodiments further can be implemented in a wide variety of operating environments, which in some cases can include one or more user computers, computing devices or processing devices which can be used to operate any of a number of applications. User or client devices can include any of a number of general purpose personal computers, such as desktop or laptop computers running a standard operating system, as well as cellular, wireless and handheld devices running mobile software and capable of supporting a number of networking and messaging protocols. Such a system also can include a number of workstations running any of a variety of commercially-available operating systems and other known applications for purposes such as development and database management. These devices also can include other electronic devices, such as dummy terminals, thin-clients, gaming systems and other devices capable of communicating via a network.", + "p0196": "Most embodiments utilize at least one network that would be familiar to those skilled in the art for supporting communications using any of a variety of commercially-available protocols, such as TCP/IP, OSI, FTP, UPnP, NFS, CIFS and AppleTalk. The network can be, for example, a local area network, a wide-area network, a virtual private network, the Internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network and any combination thereof.", + "p0197": "In embodiments utilizing a Web server, the Web server can run any of a variety of server or mid-tier applications, including HTTP servers, FTP servers, CGI servers, data servers, Java servers and business application servers. The server also may be capable of executing programs or scripts in response requests from user devices, such as by executing one or more Web applications that may be implemented as one or more scripts or programs written in any programming language, such as Java \u00ae , C, C# or C++, or any scripting language, such as Perl, Python or TCL, as well as combinations thereof. The server may also include database servers, including without limitation those commercially available from Oracle \u00ae , Microsoft\u00b0, Sybase\u00b0 and IBM \u00ae .", + "p0198": "The environment can include a variety of data stores and other memory and storage media as discussed above. These can reside in a variety of locations, such as on a storage medium local to one or more of the computers or remote from any or all of the computers across the network. The information may reside in a storage-area network familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers, servers or other network devices may be stored locally and/or remotely, as appropriate. Where a system includes computerized devices, each such device can include hardware elements that may be electrically coupled via a bus, the elements including, for example, at least one central processing unit , at least one input device , and at least one output device . Such a system may also include one or more storage devices, such as disk drives, optical storage devices, and solid-state storage devices such as random access memory or read-only memory , as well as removable media devices, memory cards, flash cards, etc.", + "p0199": "Such devices also can include a computer-readable storage media reader, a communications device , an infrared communication device, etc.) and working memory as described above. The computer-readable storage media reader can be connected with, or configured to receive, a computer-readable storage medium, representing remote, local, fixed and/or removable storage devices as well as storage media for temporarily and/or more permanently containing, storing, transmitting and retrieving computer-readable information. The system and various devices also typically will include a number of software applications, modules, services or other elements located within at least one working memory device, including an operating system and application programs, such as a client application or Web browser. It should be appreciated that alternate embodiments may have numerous variations from that described above. For example, customized hardware might also be used and/or particular elements might be implemented in hardware, software or both. Further, connection to other computing devices such as network input/output devices may be employed.", + "p0200": "Storage media and computer readable media for containing code, or portions of code, can include any appropriate media known or used in the art, including storage media and communication media, such as but not limited to volatile and non-volatile, removable and nonremovable media implemented in any method or technology for storage and/or transmission of information such as computer readable instructions, data structures, program modules or other data, including RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other medium which can be used to store the desired information and which can be accessed by the a system device.", + "p0201": "The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereunto without departing from the scope of the invention as set forth in the claims.", + "features": { + "F1": "A computer-implemented method for data integrity verification involves receiving one or more parts of a data payload from a remote computing device.", + "F2": "Each part of at least a subset of the one or more parts has a sub-part generated from it.", + "F3": "A digest is calculated for each generated sub-part based at least in part on the generated sub-part.", + "F4": "A hierarchical data structure is generated, with the calculated digests stored as nodes of the data structure.", + "F5": "A root node of the data structure is generated by concatenating each calculated digest and running a hash function on the concatenated digest.", + "F6": "A top-level digest of the data structure is determined by traversing the data structure to the root node.", + "F7": "The data payload is stored in archival data storage.", + "F8": "Validation is performed to check that the stored data payload matches a received data payload based on the determined top-level digest and a received top-level digest.", + "F9": "A data object identifier including the top-level digest is provided to a user of the archival data storage.", + "F10": "Upon receiving a request for the stored data payload, which includes the data object identifier, it is verified that the stored data payload has not changed before providing it to the user.", + "F11": "The sub-part is generated to be a first predefined size, and the received one or more parts are a second size that is an integer multiple of the first predefined size.", + "F12": "The integer size is based at least in part on a degree of branches of the data structure.", + "F13": "The data payload is stored from a first storage device to a second storage device, and verification is performed to ensure the data payload matches between both devices based on the top-level digest.", + "F14": "Repartitioning of the data payload is performed based on a second generated sub-part, and verification ensures the repartitioned data payload matches a stored data payload.", + "F15": "Verification is based on the top-level digest to check that a repartitioned data payload matches the received data payload, based on a second sub-part.", + "F16": "An instruction is provided to a remote computing device to enable the generation of the data structure and provide the identified top-level digest.", + "F17": "The one or more parts of the data payload are received as part of an electronic request from a remote computing device to store the data payload.", + "F18": "A function call is provided for a remote device to make the electronic request, which includes at least the provided function call.", + "F19": "An indication of a size for each of the one or more parts of the data payload is received.", + "F20": "A system comprises at least one memory storing computer-executable instructions and at least one processor configured to execute these instructions to perform the method described.", + "F21": "One or more computer-readable media store computer-executable instructions which configure processors to perform the described method for verifying data integrity." + } + } + } +] \ No newline at end of file