Verity AG

According to the present invention there is provide a method of creating a landmark map comprising the steps of: capturing a plurality of frame using a camera, each frame comprising an image; assigning an image number which denotes the order in which each frame was taken; extracting features from all the images in the captured frames; assigning a distinct identifier to each respective feature which was extracted from the image belonging to the frame which was first captured; for each image belonging to the respective frames which was captured after the first frame was captured, carrying out the following steps: (i) computing a distance between, the location of each respective extracted feature in that image, and the location of each respective feature extracted from image belonging to a previously captured frame; and (ii) determining if said computed distance is less than, a predefined threshold distance; and if the computed distance is not less than said predefined threshold distance then assign that extracted feature the same identifier as is assigned to the extracted feature in the image belonging to the previously captured frame; if the computed distance is greater than the predefined threshold distance, then assign a new identifier to the extracted feature; providing a list of features which comprises image numbers which denotes each of the captured frames; for each respective image number the identifiers which denote the features which were extracted from the image belonging to that frame; and for each identifier the coordinates which represent the location of the extracted feature in that image; carrying out structure-from-motion followed by repeated landmark representation merging and bundle adjustment, using the list of features, so as to create a landmark map. There is further provided an assembly having a processor which is configured to carry out said method of creating a landmark map.

The invention relates to a method for determining a state x k ( 9 ) of a localizing apparatus at a time t k , the state x k being a realization of a state random variable X k . The method comprises the following steps: a) receiving a first image ( 1 ) of a scene of interest ( 15 ) in an indoor environment ( 15 ), wherein the indoor environment ( 15 ) comprises N prearranged landmarks ( 16 ) having known positions in a world coordinate system ( 12 ), N being a natural number; b) receiving a second image ( 2 ) of a scene of interest ( 15 ) in the indoor environment ( 15 ); c) receiving a state estimate Formula {circumflex over ( )}I ( 3 ) of the localizing apparatus at the time t k ; d) receiving positions of currently mapped simultaneous-localization-and-mapping (SLAM) landmarks ( 4 ) in the scene of interest ( 15 ), wherein a map state s k comprises at least (i) the state x k of a localizing apparatus, (ii) the positions of the currently mapped SLAM landmarks ( 4 ), and (iii) the positions of the pre-arranged landmarks ( 16 ); e) determining ( 5 ) positions of features in the first image ( 1 ), being a natural number smaller than or equal to, and determining ( 5 ) an injective mapping estimate from the features into the set of pre-arranged landmarks ( 16 ); f) determining ( 6 ) positions of L SLAM features in the second image ( 2 ), and determining m SLAM features in the L SLAM features, wherein said m SLAM features are related to the n currently mapped SLAM landmarks ( 4 ), and determining ( 6 ) a SLAM injective mapping estimate from the m SLAM features into the set of the n currently mapped SLAM landmarks ( 4 ); g) using the determined injective mapping estimate and the determined SLAM injective mapping estimate to set up ( 7 ) a joint observation model as part of a state-space model, wherein the joint observation model is configured to map a map state random variable S k of which the map state s k is a realization onto a joint observation random variable Z k , wherein at the time t k , an observation z k is a realization of the joint observation random variable Z k , and wherein the observation comprises the position of at least one of the M features in the first image ( 1 ) and the position of at least one of the m SLAM features in the second image ( 2 ); and h) using ( 8 ) (i) the state estimate Formula {circumflex over ( )}I ( 3 ), (ii) the joint observation model, and (iii) the observation z k , to determine the state x k ( 9 ) of the localizing apparatus at the time t k and to update the positions of the n currently mapped SLAM landmarks. The invention also relates to a computer program product and an assembly.

According to the present invention there is provide a method a method for matching features in an image with landmark representations in a landmark map, the method comprising: providing at least one image; extracting from the image one or more features; providing a landmark map comprising a list of landmark representations, wherein each landmark representation is a representation of a respective physical landmark; creating a plurality of policies, each policy comprising at least one matching decision of the feature and the landmark representation, each matching decision having a stage cost; assigning a policy cost to each of the policies, wherein the policy cost is a function of the stage costs of the matching decisions the policy is comprised of; selecting from the collection of the policies the policy with the desired policy cost. There is further provided an assembly having a processor which is configured to carry out said method.

The invention relates to a method for determining a state x k ( 8 ) of a camera ( 11 ) at a time t k , the state x k ( 8 ) being a realization of a state random variable X k , wherein the state is related to a state-space model of a movement of the camera ( 11 ). The method comprises the following steps: a) receiving an image ( 1 ) of a scene of interest ( 15 ) in an indoor environment ( 15 ) captured by the camera ( 11 ) at the time t k , wherein the indoor environment ( 15 ) comprises N landmarks ( 9 ) having known positions in a world coordinate system ( 12 ), N being a natural number; b) receiving a state estimate x{circumflex over ( )} k ( 2 ) of the camera ( 11 ) at the time t k , c) determining ( 3 ) positions of M features in the image ( 1 ), M being a natural number; d) receiving ( 4 ) distance data indicative of distance between the M features and the corresponding M landmarks ( 9 ), respectively; e) determining ( 5 ) an injective mapping estimate from the M features into the set of the N landmarks ( 9 ) using at least (i) the positions of the M features in the image and (ii) the state estimate ( 2 ); f) using the determined injective mapping estimate ( 5 ) to set up ( 6 ) an observation model in the state-space model, wherein the observation model is configured for mapping the state random variable X k of the camera onto a joint observation random variable Z k , wherein at the time t k , an observation z k is a realization of the joint observation random variable Z k , and wherein the observation z k comprises (i) the position of at least one of the M features in the image, and (ii) the distance data indicative of distance; and g) using ( 7 ) (i) the state estimate, (ii) the observation model, and (iii) the observation z k , to determine the state x k ( 8 ) of the camera at the time t k . The invention also relates to a computer program product and to an assembly.

The invention relates to a method for controlling a light source ( 7 ), the method using (a) at least one pose estimate ( 1 ) of a camera ( 8 ) configured to capture one or more images of a scene of interest ( 13 ) which comprises at least one landmark ( 9 ), as said light source is operated to emit light which illuminates said scene of interest, (b) a landmark map ( 2 ) comprising at least 3 D location information of a plurality of landmarks comprising the at least one landmark in the scene of interest, (c) an illumination model ( 3 ) describing a relationship between an emission illumination power and reflection illumination power, wherein said emission illumination power is the power of light emitted by the light source ( 7 ) to illuminate said scene of interest, and said reflection illumination power is the illumination power of light reflected by one or more landmarks in said scene of interest and received by the camera, and (d) a predefined threshold reflection illumination power ( 4 ). The method comprises the following steps: (a) determining (5), for at least one of the plurality of landmarks, at least one optimized emission illumination power of light ( 6 ) to be emitted by the light source, and an illumination time course ( 6 ) during which the light source should be operated to emit light which has an emission illumination power which is equal to the at least one optimized emission illumination power, using (i) the at least one pose estimate ( 1 ) of the camera, (ii) the 3D location information of the at least one of the plurality of landmarks, (iii) the illumination model ( 3 ), and (iv) the predefined threshold reflection illumination power ( 4 ); and (b) operating the light source ( 7 ) to emit light which has an emission illumination power which is equal to the at least one optimized emission illumination power ( 6 ), for a time period which is equal to the determined illumination time course ( 6 ).

The invention relates to a method for determining a state x k ( 7 ) of a camera ( 11 ) at a time t k , the state x k ( 7 ) being a realization of a state random variable X k , wherein the state is related to a state-space model of a movement of the camera ( 11 ). The method comprises the following steps: a) receiving an image ( 1 ) of a scene of interest ( 8 ) in an indoor environment ( 8 ) captured by the camera ( 11 ) at the time t k , wherein the indoor environment ( 8 ) comprises N landmarks ( 9 ) having known positions and orientations in a world coordinate system ( 12 ), N being a natural number; b) receiving a state estimate Formula I ( 2 ) of the camera ( 11 ) at the time t k , wherein the state estimate ( 2 ) comprises an estimate of the pose of the camera; c) determining ( 3 ) positions of M features in the image ( 1 ), M being a natural number; and d) determining ( 6 ) the state x k ( 7 ) of the camera ( 11 ) at the time t k based on (i) observation z k at the time t k , the observation z k being a realization of a joint observation random variable z k , the observation z k comprising the positions of the M features and data indicative of distance between each of the M features and its corresponding object point in the scene of interest, respectively, and (ii) the state estimate Formula {circumflex over ( )}I ( 2 ), wherein the determining ( 6 ) of the state x k ( 7 ) comprises determining ( 4 ) an injective mapping estimate from at least a subset of the M features into the set of the N landmarks ( 9 ), and wherein the determining ( 6 ) of the state x k ( 7 ) is based on an observation model set up ( 5 ) based on the determined injective mapping estimate. The invention also relates to a computer program product and to an assembly.

The present invention relates to a method for identifying at least one candidate feature in an image of a scene of interest captured by a camera, and to a method for capturing an image, with spatial variations in image sharpness, of a scene of interest by a camera, and to a method for determining a state x k of a camera at a time t k , as well as to an assembly and two computer program products.

The system receives position information in both an internal coordinate frame and an external coordinate frame. The system uses a comparison of position information in these flames to determine orientation information. The system determines one or more orientation hypotheses, and analyzes the position information based on these hypotheses. The system may include on-board accelerometers, gyroscopes, or both that provide the measurements in the internal coordinate frame. These measurements may be integrated otherwise processed to determine position, velocity, or both. Measurements in the external frame are provided by GPS sensors or other positioning systems. Position information is transformed to a common coordinate frame, and an error metric is determined. Based on the error metric, the system estimates a likelihood metric for each hypothesis, and determines a resulting hypothesis based on the maximum likelihood or a combination of likelihoods.

The system receives position information in both an internal coordinate frame and an external coordinate frame. The system uses a comparison of position information in these frames to determine orientation information. The system determines one or more orientation hypotheses, and analyzes the position information based on these hypotheses. The system may include on-board accelerometers, gyroscopes, or both that provide the measurements in the internal coordinate frame. These measurements may be integrated or otherwise processed to determine position, velocity, or both. Measurements in the external frame are provided by GPS sensors or other positioning systems. Position information is transformed to a common coordinate frame, and an error metric is determined. Based on the error metric, the system estimates a likelihood metric for each hypothesis, and determines a resulting hypothesis based on the maximum likelihood or a combination of likelihoods.