 and infor ation, co
Image compression method with negligible and quantifiable information loss and high compression ratio
Updated Time 12 June 2019
Patent Registration DataPublication Number
US10063891
Application Number
US15/390240
Application Date
23 December 2016
Publication Date
28 August 2018
Current Assignee
UNIVERSITE DE GENEVE
Original Assignee (Applicant)
UNIVERSITE DE GENEVE
International Classification
G06K9/36,H04N19/124,H04N19/176,H04N19/91,H04N19/44
Cooperative Classification
H04N19/91,H04N19/44,H04N19/176,H04N19/124,H04N19/147
Inventor
MARTIN, ANTHONY,ZBINDEN, HUGO,SANGUINETTI, BRUNO
Patent Images
This patent contains figures and images illustrating the invention and its embodiment.
Abstract
A data compression method wherein said data comprises noise and information, comprising a data acquisition step, a precompression parameter selection step, wherein said precompression parameters are linked to an information loss, a compression step, and a storage step, the compression step being characterized in that it comprises a lossy precompression for removing some noise of the data, carried out using the precompression parameters selected in the selection step followed by lossless compression for compressing the remaining data.
Claims
1. A data compression method wherein said data comprises noise and information, comprising:
a data acquisition step, a parameter determination step allowing to determine precompression parameters which are linked to an information loss, a compression step, and a storage step, said compression step comprising a lossy precompression for removing some noise of the data followed by a lossless compression for compressing the remaining data, wherein:
said parameter determination step comprises
a system characterization step outputting the amount of image information that is lost by the compression algorithm as a function of precompression parameters and a precompression parameter selection step allowing to choose appropriate precompression parameters, said lossy precompression being carried out using the precompression parameters selected in the precompression parameter selection step, and said parameter determination step determines an upper bound on information loss and precompression parameters adapted to remove noise within said upper bound on information loss.
2. The data compression method of claim 1, wherein said parameter determination step determines said precompression parameters by establishing a plot of the information loss vs. said precompression parameters.
3. The data compression method of claim 1, wherein the system characterization step comprises:
setting the radiance of light L(x, y) to be measured, starting from L=0, acquiring a number of images through an image sensor, an amplifier and a converter, calculating an input uncertainty σ_{i}(x, y) on the data, setting a set of precompression parameters g, applying a lossy precompression followed by a lossless compression to the data, decompressing the data, calculating an output uncertainty σ_{o}(x, y) on the data, repeating the above steps by increasing L and g, reporting information loss as a function of the set of precompression parameters g, where L(x, y) is the radiance of light at point (x, y), g is the set of precompression parameters, σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression, σ_{o}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression.
4. The data compression method of claim 1, wherein the lossy precompression of the data is applied in the natural, nontransformed space of the data.
5. A data decompression method comprising
obtaining compressed data that have been compressed by the compression method of claim 1, obtaining an upper bound on information loss, decompressing the compressed data,determining an uncertainty on the decompressed data according to the following relation:
σ_{o}/L_{o}=Δ(g,L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})} where: Δ(L, g), Δ is the relative increase in uncertainty, here as a function of light hitting the pixel and precompression parameter, and on its own, d_{o }is the decompressed digital value, σ_{o}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression, σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression, L_{σo}(x, y) is the estimate of radiance L(x,y) as derived from measurement, after compression, L_{σi}(x, y) is the estimate of radiance L(x,y) as derived from measurement, before compression, g is the set of precompression parameters, and n_{j }is the number of samples.
6. The data decompression method of claim 5, wherein g is 2 and Δ(g, L_{o}) is 1.15.
7. The data compression method of claim 1, further comprising a method of decompression performed after the data compression method of claim 1 that produces compressed data, the method of decompression comprising:
obtaining the compressed data,
obtaining an upper bound on information loss, decompressing the compressed data,determining an uncertainty on the decompressed data according to the following relation:
σ_{o}/L_{o}=Δ(g,L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})} where: Δ(L, g), Δ is the relative increase in uncertainty, here as a function of light hitting the pixel and precompression parameter, and on its own, d_{o }is the decompressed digital value, σ_{o }(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression, σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression, L_{σo}(x, y) is the estimate of radiance L(x,y) as derived from measurement, after compression, L_{σi}(x, y) is the estimate of radiance L(x,y) as derived from measurement, before compression, g is the set of precompression parameters, and n_{j }is the number of samples.
8. A nontransitory computer program run on a computer or a dedicated hardware device, adapted to carry out the method of claim 7.
9. A nontransitory computer program of claim 8, wherein the dedicated hardware device comprises a FieldProgrammable Gate Array (FPGA).
Claim Tree

1od wherein said data compris s noise

thod of claim 1, wherein said parameter de erminat
 on step determines said precompression parameters by establishing a plot of the information loss vs. said precompression parameters. 3. The data compression m

thod of claim 1, wherein the system charac erizati
 n step comprises: setting the ra iance of

thod of claim 1, wherein the lossy precom ression
 of the data is applied in the natural, nontransformed space of the data. 5. A data decompression m

ub>o) is 1.15. 7. The data compressi n method of claim
 , further comprising a method of decompression performed after the data compression method of claim 1 that produces compressed data, the method of decompression comprising: obtaining the compressed data, obtaining an upper bound on information loss, decompressing the compressed data,determining an uncertainty on the decompressed data according to the following relation: σ_{o}/L_{o}=Δ(g,L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})} where: Δ(L, g), Δ is the relative increase in uncertainty, here as a function of light hitting the pixel and precompression parameter, and on its own, d_{o }is the decompressed digital value, σ_{o }(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression, σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression, L_{σo}(x, y) is the estimate of radiance L(x,y) as derived from measurement, after compression, L_{σi}(x, y) is the estimate of radiance L(x,y) as derived from measurement, before compression, g is the set of pre


5thod comprising obtaining comp essed data
 that have been compressed by the compression method of claim 1, obtaining an upper bound on information loss, decompressing the compressed data,determining an uncertainty on the decompressed data according to the following relation: σ_{o}/L_{o}=Δ(g,L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})} where: Δ(L, g), Δ is the relative increase in uncertainty, here as a function of light hitting the pixel and precompression parameter, and on its own, d_{o }is the decompressed digital value, σ_{o}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression, σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression, L_{σo}(x, y) is the estimate of radiance L(x,y) as derived from measurement, after compression, L_{σi}(x, y) is the estimate of radiance L(x,y) as derived from measurement, before compression, g is the set of precompression parameters, and n_{j }is t

e number of samples. 6. The data decompressi n metho
 of claim 5, wherein g is 2 and Δ(g, L<

8compression parameters, and n_{j }is the number of samples. 8. A nontransitory computer program run on a computer or a de

icated hardware device, adapted to carry out the method
 f claim 7. 9. A nontransitor computer

Description
RELATED APPLICATION
The present application claims priority to European Patent Application No. 15202608.4, filed Dec. 23, 2015, the disclosure of which is hereby incorporated by reference herein in its entirety.
FIELD OF THE INVENTION
The present disclosure relates to data processing, and more specifically to the encoding and compression of image data.
BACKGROUND OF THE INVENTION
The amount of images produced worldwide is in constant growth. Photographic and video data, by its nature, consumes a large part of available digital resources, such as storage space and network bandwidth. Image compression technology plays a critical role by reducing storage and bandwidth requirements. There is therefore a need for high compression ratios whilst ensuring a quantifiably negligible loss of information.
Imaging is the technique of measuring and recording the amount of light L (radiance) emitted by each point of an object, and captured by the camera. These points are usually laidout in a twodimensional grid and called pixels. The imaging device records a measured digital value d_{adc}(x, y) that represents L(x, y), where (x, y) are the coordinates of the pixel. The ensemble of values d_{adc}(x, y) forms a raw digital imageM. Typically M consists of several tens of millions pixels and each d_{adc}(x, y) is coded over 8 to 16 bits. This results in each raw image requiring hundreds of megabits to be represented, transferred and stored. The large size of these raw images imposes several practical drawbacks: within the imaging device (photo or video camera) the size limits the image transfer speed between sensor and processor, and between processor and memory. This limits the maximum frame rate at which images can be taken, or imposes the use of faster, more expensive and more power consuming communication channels. The same argument applies when images are copied from the camera to external memory, or transmitted over a communication channel. For example:
 transmitting raw “4K” 60 frames per second video in real time requires a bandwidth of 9 Gbps; and storing one hour of such video requires 32 Tbits.
 transmitting a single 40 MPixel photograph from a cube satellite over a 1 Mbps link takes 10 minutes.
 backing up a 64 GByte memory from a photographic camera over a 100 Mbps Internet connection takes more than one hour.
To alleviate requirements on communication time and storage space, compression techniques are required. Two main classes of image compression exist:
 A. Lossless image compression, where digital image data can be restored exactly, and there is zero loss in quality. This comes at the expense of very low compression ratios of typically 1:2 or less.
 B. Lossy image compression, where the compressed data cannot be restored to the original values because some information is lost/excised during the compression, but which can have large compression ratios of 1:10 and higher. These compression ratios however come at the expense of loss of image quality, i.e. distortion, and the creation of image compression artifacts, i.e. image features that do not represent the original scene, but that are due to the compression algorithm itself.
Lossy image compression methods, such as those in the JPEG standard or used in wavelet compression, create distortion and artifacts. Distortion is the degree by which the decoded image Q, i.e. the result of compressing and decompressing the original image, differs from the original image M, and is usually measured as the rootmeansquare of the differences in the values of each corresponding pixel between Q and M. Lossy compression typically also introduces artefacts, which are a particularly bad type of distortion that introduces image features in Q, i.e. correlations between different pixels, that are not present in M. Examples of artefacts are the “block” artifacts generated by block image compression algorithms like JPEG, but also ringing, contouring, posterizing. These artifacts are particularly nefarious, as they can be mistaken for image features. Lossy image compression algorithms, as described for example in prior art documents [1] and [2] cited below, typically consist of several steps. First, the image is transformed into a representation where the correlation between adjacent data point values is reduced. This transformation is reversible, so that no information is lost. For example, this step would be similar to calculating the Fourier coefficients, i.e. changing an image that is naturally encoded into a position space, into a special frequency space. The second step is referred to as quantization. This step truncates the value of the calculated coefficients to reduce the amount of data required to encode the image (or block, consisting of e.g. 16×16 pixels). This second step is irreversible, and will introduce quantization errors when reconstructing the image from the aforementioned coefficients. The quantization error will cause errors in the reconstructed image or block. Predicting or simulating what the error would be for any image or block is technically impossible, due to the extremely large number of values that such image or block may take. For example, a 16×16 block, with 16bits per pixel, may take 2^{16×16×16}≅10^{4932 }different values, impossible to test in practice. If the transformation is not restricted to blocks, but rather is a function of the entire image, the number of possible values becomes considerably larger.
More particularly, in standard lossy image compression techniques (e.g. document [1]), a lossless “image transformer” is first applied to the image, resulting in transformed image, where each datapoint is a function of many input pixels and represents the value of a point in a space that is not the natural (x,y) position space of the image, but rather is a transformed space, for example a datapoint might represent a Fourier component. A lossy “image quantizer” is applied on the transformed image as a second step. The amount of information loss cannot be accurately quantified in standard lossy image compression techniques for the above mentioned reason of the extremely large number of values that such image or block may take and is additionally complicated by the fact that the lossy operation is applied on a space that is not the natural image space.
Attempts have been made at characterizing the quality of the output of image compression algorithms. These efforts have typically focused on characterizing the quality of the reconstructed compressed image with respect to the human visual system, as reviewed in documents [2, 3, 4, 5] described below. Several metrics have been devised to characterize the performance of the compression algorithm, however, these metrics have significant drawbacks. One such drawback is that as they relate quality to the human visual system and human perception, which are highly dependent on the subject viewing the image, on the viewing conditions (e.g. eyetoimage distance, lighting of the image, environment lighting conditions, attention, angle, image size) as well as on the image rendering algorithms (e.g. debayering, gamma correction) and characteristics of the output device, such as display, projector screen, printer, paper etc. A second drawback of characterizing the quality of image compression algorithms with respect to the human visual systems or models thereof is that for such methods to be relevant, no further image processing must take place after image compression, as such processing would make some unwanted compression artifacts visible. For example, a compression algorithm might simplify dark areas of an image, removing detail, judging that such detail would not be visible by the human eye. If it is later decided to lighten the image, this detail will have been lost, resulting in visible artifacts. Yet another problem of the abovementioned methods is that they are unsuitable for applications not aimed solely at image reproduction for a human observer, as for example are images for scientific, engineering, industrial, astronomical, medical, geographical, satellite, legal and computer vision applications, amongst others. In these applications data is processed in a different way as by the human visual system, and a feature invisible to the untrained human eye and therefore removed by the abovementioned image compression methods, could be of high importance to the specific image processing system. For example, inappropriate image compression of satellite imagery could result in the addition or removal of geographical features, such as roads or buildings.
More recently, attempts at a more quantitative approach on the information loss in digital image compression and reconstruction have been made [6], in particular trying to examine information loss given by the most common image compression algorithm: Discrete Wavelet Transform (DWT), Discrete Fourier Transform (DFT) and 2D Principal Component Analysis (PCA). Several “quantitative” metrics are measured on several images compressed with several algorithms, for example quantitative measures of “contrast”, “correlation”, “dissimilarity”, “homogeneity”, discrete entropy, mutual information or peak signaltonoise ratio (PSNR). However, the effect of different compression algorithms and parameters on these metrics highly depends on the input image, so that it is not possible to specify the performance of an algorithm, or even to choose the appropriate algorithm or parameters, as applying them on different images gives conflicting results. A further drawback of the methods described in [6], is that although these methods are quantitative in the sense that they output a value that might be correlated with image quality, it is unclear how this number can be used: these numbers cannot be used as uncertainties on the value, the compression methods do not guarantee the absence of artifacts, and it is even unclear how these numbers can be compared across different compression methods and input images. Also, these quantitative methods do not distinguish signal from noise, so that for example, a high value of entropy could be given by a large amount of retained signal (useful) or retained noise (useless).
The impossibility for the abovementioned methods to achieve compression with quantifiable information loss arises from the fact that the space in which the algorithm looses data is very large (e.g. number of pixels in the image or block times the number of bits per pixel), and cannot therefore be characterized completely, as we have mentioned earlier.
Imageprocessing systems that act independently on individual (or small number of) pixels have been known and used for a long time. These systems typically provide a “lookup table” of values so that each possible input value is associated with an “encoded” output value. The main purpose of such processing has been to adapt the response curves of input and output devices, such as cameras and displays respectively. The functions used to determine the lookup tables have typically been logarithmic or gamma law, to better reflect the response curve of the human visual system as well as the large number of devices designed to satisfy the human eye, starting from legacy systems like television phosphors and photographic film, all the way to modern cameras, displays and printers that make use of such encodings. Although not their primary purpose, these encodings do provide an insignificant level of data compression, such as for example encoding a 12bit raw sensor value over 10bits, thus providing a compression ratio of 1.2:1, as described, for example in [7].
REFERENCES
 [1] W. M. Lawton, J. C. Huffman, and W. R. Zettler. Image compression method and apparatus, 1991. U.S. Pat. No. 5,014,134.
 [2] V. Ralph Algazi, Niranjan Avadhanam, Robert R. Estes, and Jr. Quality measurement and use of preprocessing in image compression. Signal Processing, 70(3):215229, 1998.
 [3] Ahmet M Eskicioglu and Paul S Fisher. Image quality measures and their performance. Communications, IEEE Transactions on, 43(12):29592965, 1995.
 [4] Philippe Hanhart, Martin Rerabek, Francesca De Simone, and Touradj Ebrahimi. Subjective quality evaluation of the upcoming hevc video compression standard. 2010.
 [5] Weisi Lin and C.C. Jay Kuo. Perceptual visual quality metrics: A survey. Journal of Visual Communication and Image Representation, 22(4):297312, 2011.
 [6] Zhengmao Ye, H. Mohamadian, and Yongmao Ye. Quantitative analysis of information loss in digital image compression and reconstruction. In Communication Software and Networks (ICCSN), 2011 IEEE 3rd International Conference on, pages 398402, May 2011.
 [7] Lars U. Borg. Optimized log encoding of image data, 2012. U.S. Pat. No. 8,279,308.
 [8] T. A. Welch. A technique for highperformance data compression. Computer, 17(6):819, June 1984.
 [9] I. Matsuda, H. Mori, and S. Itoh. Lossless coding of still images using minimumrate predictors. In Image Processing, 2000. Proceedings. 2000 International Conference on, volume 1, pages 132135 vol. 1, 2000.
SUMMARY
In view of the above, it is therefore an object of the invention to provide a data compression method that permits to compress data with a high compression ratio with minimal information loss and to provide a technique for determining the information loss or the uncertainty with respect to the compression ratio.
A further object of the invention is to provide a decompression method adapted to decompress the data compressed by the compression method of the invention.
A further object of the invention is a data processing method comprising the compression method and the decompression method.
Here are described the embodiments of an image compression system and method that have a number of advantages:
 Precise estimation of information loss at each chosen compression ratio.
 A specific, quantifiable, bound for the compression ratio at which information loss remains negligible.
 High compression ratio (e.g. 8:1) with negligible information loss.
 No compression artifacts.
 Both the pixel value and uncertainty (“error bar”) on the pixel value can be recovered from the compressed image data and used in subsequent processing steps.
 Allows the user to choose the desired compromise between information loss and level of compression.
 The system can be characterized for all possible photographs, so that the amount of information lost can be predicted for any possible photograph.
To achieve the above advantages, the embodiments presented here distinguish the amount of image information from the amount of image data. Information is the useful knowledge acquired about the photographed object by means of the camera. Data is the digital representation of the measurement result, and contains information as well as noise and redundancy. In general, the amount of data captured will be much larger than the amount of information, so that it is possible to eliminate a large amount of data whilst eliminating only very little information.
Before going into its details in the next section, an overview of an embodiment is provided. First, the imaging system and compression method (e.g. sensor and processing) are characterized, for example by calibration, by simulation or theoretically, to determine the optimal precompression parameters. In contrast to the standard lossy image compression techniques, an appropriatelytuned lossy precompression, or quantizer, is applied directly on the acquired raw image data, as a precompression step. The amount of information lost by this quantizer can be calculated or measured, and appropriate parameters chosen to guarantee that such information loss remains negligible. Precompression removes most of the noise, which cannot, by its nature, be losslessly compressed, however, thanks to the properly chosen parameters, this step only removes a negligible amount of information. The appropriatelyquantized image data, that now contains mostly information and little noise, is fed to a lossless compression algorithm as a second step. This algorithm will work very effectively, as more redundancy can be found in this clean data than could be found in the raw image sensor data that contained a large amount of noise. As now the second step is lossless, it will not reduce the amount of information present in the image and therefore does not need to be characterized for information loss.
The amount of information lost can be characterized as the increase in the uncertainty (i.e. the standard deviation, or “error bar”) in the measured value of the radiance after the application of the compression and decompression algorithm, and define that this loss is negligible when the uncertainty on pixel values after decompression remains smaller than the standard deviation between the pixel values of consecutive photographs of the same subject.
The invention relates to a data compression method wherein the data comprises noise and information, comprising a data acquisition step, a precompression parameter selection step wherein said precompression parameters are linked to an information loss, a compression step, and a storage step, said compression step being characterized in that it comprises a lossy precompression for removing some noise of the data, carried out using the precompression parameters determined in the calibration step followed by lossless compression for compressing the remaining data.
Preferably, the compression method comprises a parameter determination step for determining precompression parameters by establishing a plot of the information loss vs. said precompression parameters.
Preferably, the parameter determination step determines precompression parameters adapted to remove most of the noise.
Preferably, the parameter determination step comprises a system characterization step outputting the amount of image information that is lost by the compression algorithm as a function of precompression parameters and a selection step, wherein a user chooses appropriate precompression parameters.
Preferably, the system characterization step comprises
 setting the amount of light L to be measured, starting from L=0 (800).
 acquiring a number of images (801) through an image sensor (306), an amplifier (310) and a converter (312).
 calculating an input uncertainty σ_{i}(L)/L on the data,
 setting a precompression parameter g,
 applying a lossy precompression followed by a lossless compression to the data;
 decompress the data;
 calculate output uncertainty on the data;
 repeat the above steps by increasing L and g;
 report information loss as a function of precompression parameter g.
The invention also relates to a data decompression method comprising
 obtaining compressed data, that have been compressed by the compression method described above,
 obtaining bound on information loss
 decompressing the compressed data,
 determining an uncertainty on the decompressed data according to the following relation:
σ_{o}/L_{o}=Δ(g,L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})}
where:
Δ(L, g), Δ is the relative increase in uncertainty, here as a function of light hitting the pixel and precompression parameter, and on its own,
d_{o }is the decompressed digital value,
σ_{o }(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), after compression,
σ_{i}(x, y) is the uncertainty (standard deviation) associated with the measurement estimate of the radiance L(x, y) at point (x, y), before compression,
L_{σo}(x, y) is the estimate of radiance L(x,y) as derived from measurement, after compression,
L_{σi}(x, y) is the estimate of radiance L(x,y) as derived from measurement, before compression,
g is the precompression parameter, and
n_{j }is the number of samples.
The invention also relates to a data processing method comprising the compression method followed by the decompression method described above.
The invention also relates to a computer program run on a computer or a dedicated hardware device, like a FPGA, adapted to carry out the data processing method described above.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 shows a schematic overview of how an image to be treated by a method according to the present invention is typically acquired.
FIG. 2 shows a schematic overview of the typical steps of acquiring raw image data of an image to be treated by a method according to the present invention.
FIG. 3 shows a schematic overview of the system calibration step performed by a method according to the present invention in order to allow for determination of precompression parameters.
FIGS. 4A, 4B and 4C show, by way of corresponding schematic overviews, a data processing method according to the present invention which is preferably split into 3 phases, namely parameter determination, data compression, and data decompression.
FIG. 5 shows the quantified uncertainty increase (information loss) bound as a function of radiance L, the units being normalized to the mean number of photoelectrons n.
FIG. 6 shows the uncertainty/information loss as a function of an example of a precompression parameter g.
DETAILED DESCRIPTION
Imaging System
How an image is typically acquired is described more in detail as follows. This is illustrated in FIGS. 1 and 2. An imaging device aims at measuring (estimating) the amount of light emitted by a specific area (302) of an object. This amount of light shall be referred to as the radiance L(x, y) illuminating a pixel (308) with coordinates (x, y) on the image sensor (306). Light from this area is collected by an imaging system (304), such as a camera objective, and focused on a specific pixel (308) of the image sensor (306). During the camera's exposure time T, a number of photons γ(x, y) will impinge on the pixel (308), and generate a number of photoelectrons n(x, y). These electrons (a charge) will typically be translated to a voltage v(x, y) by an amplifier (310). This voltage is digitized by an analoguetodigital converter (ADC) (312) that outputs a digital valued_{ADC}(x, y). The raw image data (314, 212) is the ensemble of values d_{ADC}(x, y) for all (x, y) comprising the image. From d_{ADC}(x, y) it is possible to estimate L_{σi}(x, y), i.e. the value of the radiance L(x, y) with an uncertainty σ_{i}(x, y). The uncertainty is given by noise, such as shot noise and read noise. If the sensor, amplifier and ADC are linear, the mean value d_{ADC}(x, y) can be related to the mean value of the number of photoelectrons n by proportionality constant ζ and an offset constant c:
d_{ADC}(x,y)=ζ(n(x,y)+c (EQ1)
Similarly, the mean number of photoelectrons n is proportional to the number of photons γ(x, y) that impinge on the pixel during the integration time T with a proportionality constant called the quantum efficiency QE:
n(x,y)=QE×γ(x,y) (EQ2)
The number of photons γ(x, y) is proportional to the radiance L(x, y), i.e. the amount of luminous power emitted by the object by unit area and solid angle. γ(x, y) is also proportional to other constants, such as the observed area A (function of the imaging system (304) and pixel size), observed solid angle Ω (function of the imaging system (304)), exposure time T, and inversely proportional to photon energy E=hc/λ where h is Planck's constant and λ is the wavelength of the incident light. This can be summarized as:
Substituting Equation (EQ3) into Equation (EQ2) and the resulting equation into Equation (EQ1), one obtains:
Setting Z=ζQEAΩT/E,
d_{ADC}(x,y)=ZL(x,y)+c (EQ5)
So that L(x, y) can be directly evaluated from d_{ADC}(x, y) as:
And on for a single shot (i.e. without averaging) one may evaluate L_{σi}(x, y) as:
Equation (EQ7) serves as an example for a system that is very linear. If this is the case, and the sensor, amplifier and converter result in a shotnoise limited measurement, the number of photoelectrons will be Poissondistributed, so that the uncertainty in photoelectron number will be Δn=√{square root over (n)}, and the relative uncertainty
If the sensor, amplifier or ADC presents nonlinearity, it is necessary to calibrate them, for example by creating a lookuptable LUT_{d→L }and LUT_{d→σi }that associates a value of L and σ_{i }to each possible value of d_{ADC}. How this is done in practice is described as follows.
In conclusion, the raw data will be the recorded digital value d_{adc}(x, y), the information acquired about the object radiance L(x, y) will have value L_{σi}(x, y) with uncertainty σ_{i}(x, y). When image data undergoes compression and decompression, resulting in the possibility to infer an output L_{σo}(x, y) with an uncertainty σ_{o}(x, y), it is then possible to quantify the amount of information lost by looking at the relative increase in uncertainty: σ_{o}(x, y)/σ_{i}(x, y), i.e. the relative increase in error bars.
An overview of a preferred embodiment of the method of the present invention will now be described with reference to FIG. 3 as well as 4A, 4B, and 4C.
This preferred embodiment is a data processing method which is preferably split into 3 phases as depicted in FIGS. 4A, 4B and 4C.
1) The first phase also called “parameter determination” (250) and represented in FIG. 4A serves to determine precompression parameters (210) that achieve a high compression ratio while guaranteeing that the information lost is below a specific bound (208).
2) The second phase is the compression method of the present invention represented in FIG. 4B which compresses image data using the parameters (210) determined in the parameter determination phase (250). The compressed data are transmitted and/or stored (220).
3) The third phase is the decompression method of the present invention represented in FIG. 4C which recovers compressed data (222) from the transmission channel or storage (220), as well as the bound on information loss (224). This information is used to obtain decompressed data (228), as well as the uncertainty on this data (230). Both of these can be used for processing (232) and display or other usage (234).
It is important to note that even if the data processing method of the invention comprises these three phases, each single phase can be carried out independently from each other. Such that the present invention comprises a parameter determination method, a compression method, a decompression method as well as a data processing method comprising one or more of these three methods.
We will now describe the first phase of parameter determination.
Parameter Determination Phase for System Calibration
As shown in FIG. 4A, in order to carry out a system calibration allowing to determine the precompression parameters and the bound on the information loss, first, the camera system is characterized (200) to determine the amount of image information that is lost (202) by the compression algorithm as a function of precompression parameters. This characterization is described in detail here below with reference to FIG. 3. In a simple exemplary embodiment, that works well for shotnoise limited, linear systems, the precompression parameter will be a single variable g taking values between 1 (no compression and data loss) and infinity (high compression and all data lost). This parameter will be used in the precompression algorithm described in the compression phase detailed below.
A sample output of this step is shown in FIG. 6 representing a plot of the uncertainty/information loss with respect to parameter g. With this data (202) in hand, and according to the preferences of the user (204), appropriate compression parameters are chosen. In the instance of FIG. 6, for example, the precompression parameter g can be increased up to a value of 2, with the amount of information lost remaining below the negligible loss bound. FIG. 5 shows the quantified uncertainty increase (information loss) bound (208) for values of L between 0 and 500 (here the units are normalized to the mean number of photoelectrons n).
In the description below, the precompression parameters is a single parameter relating to a predefined precompression function; however, in general, it may be represented by multiple parameters, or the lookuptables LUT_{d→L }and LUT_{d→σi}. The selected parameters and associated uncertainty increase bounds are saved to be reused in the next phase: Compression.
System Characterization
An embodiment of this procedure is shown in FIG. 3. System characterization evaluates the amount of information lost by the compression system as a function of compression parameter and input radiance L.
For clarity reason, a list of the symbols used here is available at the end of the present specification.
First, the amount of light to be measured is set, starting from L=0 (800), then, a number of images is acquired (801). This can be done using the real image acquisition system, by simulating such a system, or by theoretical modeling. In this example, the data arising from a shotnoise limited linear sensor is simulated. Assuming linearity means that the number of photoelectrons n will be proportional to the radiance L to be measured (at constant T, Ω, QE, A, E): n=LZ/ζ. For simplicity of the simulation, ζ is set equal to 1 and c=0 so that d_{ADC}=n. For a shotnoise limited system, n will be distributed according to a Poisson distribution N(LZ/ζ)=N(n) that has mean n, variance n and standard deviation √{square root over (n)}. With the above simplifications, the relative error σ_{i}/L_{i} will be equivalent to the relative uncertainty in the measurement of the number of photoelectrons √{square root over (n)}/n=1/√{square root over (n)}. To simulate sensor data in (801) for a specific value of n, we generate a number of samples n_{1 }issued from N(n). Instead of simulating data, (801) may acquire data from the imaging system.
The relative input uncertainty σ_{i}(L)/L is calculated or numerically computed as rootmeansquare (RMS) difference between each L_{i}, the measured value of L, and the actual (setpoint) value of L, divided by L itself (802).
A precompression parameter, or set of precompression parameters is chosen (804). In our simple example, a single parameter g is used. However, more in general, this could be a lookuptable (LUT) associating an output value of L_{o }to each possible value of L_{i}.
Lossy precompression is then applied to each input measured value of L_{i }(808). In this embodiment, the precompressed digital value d_{c }is obtained from the function
where n is the number of photoelectrons on that sample, i.e.
with respect to the ADC output, n=(d_{ADC}−c)/ζ. The doubleline “brackets” symbolize taking the nearest integer. In this case, g represents the reduction factor in the number of bits required to encode each possible number of photoelectrons. For example, encoding a number of photoelectrons between 0 and 2^{16}−1=65535 requires 16 bits. For g=1, no precompression would arise, and 16 bits would be required. For g=2, the number of encoding bits is reduced by a factor of 2, so that the result of the precompression can be encoded over 8 bits. More in general, any LUT can be applied.
A lossless compression algorithm may then be applied (810), either on a pixelbypixel basis, or on larger blocks, or on the entire precompressed image. The data to which this lossless compression is applied is called F. As after the lossy precompression is applied, the amount of data is reduced, but also, the entropy of such data is reduced (noise is removed) so that lossless compression will be very effective. In our example, we have used several algorithms, ranging from the universal and simple LempelZivWelch (LZW) [8] to imagecontentbased algorithms such as minimumrate predictors (MRP) [9]. The algorithm here can be chosen at will, as being lossless, it will not lose any information, and will not affect the uncertainty of the image. In some instances, if compression speed is of the essence, this step may be a simple “passthrough” not affecting the precompressed data at all.
In the next step (812), the output f of this lossless compression is decompressed to obtain F, then, the inverse function of the lossy compression is applied (or the inverse LUT). For example, the inverse of
is d_{o}→d_{c}^{g}.
From the decompressed output d_{o }it is possible to calculate a value L_{o }for the radiance, e.g. L_{o}=d_{o}ζ/Z. In (814) the error that was introduced as the difference L_{o}−L_{i }is calculated. This error can be estimated for several samples of the acquired image data, and the rootmeansquare (RMS) of this error calculated as σ_{o}(L, g), and compared to the input error (uncertainty) σ_{i}(L) to obtain a welldefined, properly quantified, relative increase in uncertainty Δ(L, g)=σ_{o}(L, g)/σ_{i}(L). The compression ratio r can also be calculated as the size of the original image data divided by the size of output f of the lossless compression.
By repeating the above process for a number of image acquisition samples, for a number of compression factors g (816, 824) and for a number of Radiances, integration times or generated photoelectrons (818, 822), one obtains a characterization of the compression system (820), i.e. the amount of information lost Δ(L, g) as a function of precompression parameters g and pixel radiance L (or equivalently, photoelectron number n or pixel value d_{c }or d_{o}, as all these are relate by known equations).
Sample results of this characterization are shown in FIG. 5, where g is varied at fixed L, and 6, where L is varied such as to produce between zero and 500 photoelectrons n, and the precompression is held fixed at g=2. The results show that Δ(L, g)=σ_{o}(L, g)/σ_{i}(L)=1.15 across the range for all these settings. This is smaller than what was defined as the bound for negligible information loss, i.e. the RMS of the difference between consecutive measurements (“photographs”) of the same pixel, of exactly the same subject taken with an ideal shotnoiselimited sensor, that is √{square root over (2)}≅1.41.
Testing this precompression factor g=2, and therefore Δ=1.15 on an image of a 10 EUR note, acquired with an ATIK383L scientific camera, the compression factors were as follows:
Original image file: 5,164,276 bytes
Lossless compressed original file (ZIP): 4,242,951 bytes.
Lossy Precompressed image: 2,580,219 bytes
Lossless compressed precompressed image (ZIP): 1,015,808 bytes
Lossless compressed precompressed image (MRP [9]): 544,336 bytes
To give a compression ratio of 9.5:1 and increasing the uncertainty by a factor of only 1.15. FIG. 6 plots both the errorbars with the original uncertainty σ_{i }and with the, barely distinguishably larger, output uncertainty σ_{o}, from which it is evident that this uncertainty increase is negligible.
It has to be noted that, when keeping parameters A, Ω, T, E constant, shot noise will manifest itself with the number of photoelectrons n following a Poisson distribution with mean and variance n and with standard deviation σ_{i}=√{square root over (n)}.
Once, the system characterization is achieved and the precompression parameters are available, the user can select the required parameters and continue with the compression phase.
Compression Method
An embodiment of this compression method is illustrated in FIG. 4B (252). Precompression parameters (210) g and associated uncertainty increase bounds (208) Δ are taken from previous phase (250) “parameter determination”. In this example, g=2, Δ=1.15, however it may be different depending on the parameter estimation results. Raw image data is first acquired (212, 314), either from the image processing pipeline or from disk/memory. Then, a lossy precompression is applied (214) in the same way as described in step 808 of the section “System characterization”, using precompression parameter g (210). Precompressed data is then losslessly compressed in step (216) such as to yield compressed image data (218) as described in step 810 of the section “System characterization”. Compressed data is then stored or transmitted (220). The bound on information loss (208) may also be included into the stored/transmitted data for future use, or may be a predefined parameter for the system.
Once, the compression is achieved and the compressed data are stored or transmitted (that actually can be considered as a mobile storage in a channel), the user can decide to decompress the data to recover the data and therefore pass to the last phase that is the decompression method.
DeCompression Method and Usage
An embodiment of this method is illustrated in FIG. 4C (254) “Decompression and usage”. First, the compressed data (222) are obtained from the transmission link or storage (220). Ideally, these data were compressed by the above compression method. The bound on information loss (224) is also obtained from the transmission link or storage, but could also be a predefined part of the system. This information is fed into a decompression routine (226) that has already been described in step 812 of the section “System characterization”. This step will output decompressed data (228), and the uncertainty on such decompressed data (230).
As a matter of fact, using the example of the compression and decompression functions described in section “System characterization”, the decompressed image data uncertainty (230) is calculated as σ_{o}/L_{o}=Δ(g, L_{o})×σ_{i}/L_{i}=1.15×1/√{square root over (n)}=1.15×1/√{square root over (d_{o})} where d_{o }is the decompressed data output for that pixel and Δ is preferably 1.15.
This data (228) and, optionally, the associated uncertainty (230), can then be used to effectively process the image (232), resulting in data to be output to display, printing, or further analysis or processing (234).
TABLE OF SYMBOLS
Great research starts with great data.
Use the most comprehensive innovation intelligence platform to maximise ROI on research.
More Patents & Intellectual Property
 Image processing system with conditional coding and method of operation thereof
 Enterprise Patent & IP Solutions
 Improve R&D Innovation
 Intellectual Property (IP) Tools
 IP & Patent Strategies
 Market Intelligence for Innovation
 IP Data API
 Chemical Structure Search
 DNA Sequence Search
 Free Intellectual Property Courses
 IP & Patent Glossary