I've not had time to review the code, but It sounds like you guys are on the right track. You now know more about the pie backend that most people, so you are in the best position to judge the common vs separate backend issue. Either is choice is acceptable. Regarding the license, the sane exception specifically allows that the exception be removed by later authors. The only question would be another backend that chooses to use your IR library. Would that backend also have to remove the exeption? Probably so, but this is a gray area.
allan On Thu, Aug 23, 2012 at 2:02 PM, Vleeshouwers, J.M. <J.M.Vleeshouwers at tue.nl> wrote: > Hi Michael, > > Good to see you picked it up again! > > Regarding your 2nd question: I think we should create a separate backend. > I've got two reasons for that: 1) the patch makes the pie backend a very > large and complex unit, and 2) I don't want to have to worry about breaking > the existing pie functionality every time we modify the usb part. I hope you > and Klaus agree. > > I did some work on this track, although it is not complete. I took your code > (version fall 2011) apart and I am now slowly getting all pieces to work > again (in a way I understand). The major thing that does not work yet is > sane_read(). Your code is based on the index-format, but I think the > pixel-format should be much more practical, since the SANE standard expects > RGB pixels to be returned from sane_read(). > I divided the code into four sections: > a. Low-level functions which communicate with the device on a byte-level. > These implement the usb-wrapper but only handle byte-arrays, no > interpretation takes place here. This section is complete and tested. > b. Mid-level scanner functions for scanner commands, which take a meaningful > struct as an argument (such as mode_select() which takes a 'struct Mode' as a > parameter) and return a 'struct Status' which includes sense data if an error > occurs. These functions automatically issue a REQUEST SENSE command if a > command returns CHECK SENSE, and repeat a command if the device is reports a > busy state. No reason to take care of this anymore at a higher level, which > is practical. This section is almost complete, and quite easy to extend. > Klaus' device has commands for moving to the next and previous slide, these > are not implemented (not yet). > c. Top-level auxiliary functions, e.g. for initialising options, correcting > shading, etc. A lot still missing. > d. Top-level SANE interface functions. Missing sane_read() and > sane_get_parameters(), the rest implemented according to the procedure of the > Windows driver. Not much tested yet. > > I see you managed to find and implement dust/scratch removal algorithms. That > is very welcome! I hope we can get a positive answer to your third question. > It looks like you chose to implement this in the backend, which is practical > since it does not break the SANE standard. This might be the best solution > for now. > This makes our wishlist a bit smaller as far as I am concerned. Remains: > - react to a button press (use scanbuttond daemon? it that the solution we > want?) > - export infrared data (for other uses than dust/scratch removal; maybe > modify the scanimage frontend and stiff.c to allow RGBI TIFF image files to > be exported?) > > So how do we proceed? > - We have got two pie-patched backends (Klaus' and yours) which I assume > should be merged > - We have got a stand-alone backend in an incomplete state (see attachment) > We should decide on a common starting point. If we decide for a separate > backend, I would prefer the four-section structure. You and Klaus are > familiar with software development, at least more than I am, so we should use > that. > > Yours, > > Jan > > -----Original Message----- > From: Michael Rickmann [mailto:mrickma at gwdg.de] > Sent: dinsdag 21 augustus 2012 22:27 > To: sane-devel at lists.alioth.debian.org; Vleeshouwers, J.M. > Subject: Reflecta ProScan / Crystalscan 7200 PIE film scanner update > > It has become a bit silent about these scanners since Jan Vleeshouwers' > post 8 months ago ( > http://lists.alioth.debian.org/pipermail/sane-devel/2011-December/029337.html > ). Finally, I had some time to collect what else Jan and I had found out last > year and to put it into a patch against the pie backend. Both scanners > identify as PIE SF Scanners. They share the same USB id > 0x05e3:0x0145 use a SCSI over USB protocol and can be distinguished by byte > 0x74 in the INQUIRY return block. Support for the ProScan is quite reasonable > now, for the Crystalscan presumably less because calibration (see below) was > mostly guessed from what Jan had snooped and communicated. I tested with > scanimage and XSane. Currently, Klaus Kaempf seems to work on a PIE MF > Scanner ( > http://lists.alioth.debian.org/pipermail/sane-devel/2012-July/030092.html ) > which is more advanced but presumably not so much different in protocol. > What can the backend do with the SF scanners hardware? > 1) scan at resolutions from 20 to 3600 dpi, increment 1 > 2) read RGB and single pass RGBI data > 3) read at 8 and 16 bit [recommended] colour depth > 4) calibrate and read in normal and quality [recommended] mode This is what > could be detected in snoops using Cyberview and Silverfast under Windows. > Silverfast does not support normal mode. Calibration and shading correction > is done with the help of software. > What can the backend do by processing the data read? > 1) option: clean the infrared colour plane from red spectral overlay > 2) option: use the infrared data to remove dirt, output RGB > 3) option: simple film grain smoothening (just a triangular blur) > 4) option: crop using sanei_magic except for cleaned scans of positive film > where an internal routine detects faster and more correctly > 5) options: simple colour conversions, constant gamma, invert Scanned > positive film really looks good, negative film bad due to still missing > colour profiles. Also the options of Xsane for negative film do not improve > things. Scanning a 36x24 mm slide at 3600 dpi takes about 88 sec in RGB, 102 > sec in RGBI with infrared discarded, 110 sec in RGBI with cleaning and > cropping. If one compares these numbers with the ones at > http://www.filmscanner.info/en/ReflectaProScan7200.html it seems that the > backend handles the scanner correctly. > > What can the backend do in spite of the USB additions? Recently I got hold of > a 12 year old Adlib JetScan 630 SCSI flatbed scanner which is supported by > the pie backend. The scanner still scans at 600 dpi (its > maximum) at ?nomal? and ?fine? speed; colour, grayscale, halftone and lineart > are ok. However at high resolutions and ?pro? speed I get hardware > ?integration time adjustment failure?s ?too dark? or ?too light?. I get the > same errors with SANE-Backends-1.0.22 as included in Ubuntu 12.04 64-bit. I > guess, it is the age or a limitation of this kind of scanner. So I think that > the direct SCSI part of the backend has not deteriorated. > > Description of the added code: I have added three files pie_usb.h contains > basic definitions for the USB code sanei_ir.h, sanei_ir.c contain code to > remove dirt with help of the infrared plane. I think that this code can > migrate to sanei_magic because it may become useful also for other backends. > There is still a license problem. I have adapted three threshold routines > from M. Emre Celebi's Fourier_0.8 project ( > http://sourceforge.net/projects/fourier-ipal/ ) which is licensed under the > GNU General Public License Version 2. > Changed files: > sanei_magic.c : sanei_magic_crop can crop images with >8 bit colour depth now > pie_scsidefs.h : mainly added support for PIE vendor specific SCSI commands. > pie.c : really a lot. All pure USB functions are prefixed with pie_usb_, all > other functions with pie_ (renamed a few) or sane_. The SCSI over USB > emulation runs via pie_usb_scsi_wrapper which branches into low level USB > code. All other functions talk SCSI. As pie_usb_scsi_wrapper uses the same > arguments as sanei_scsi_cmd functions can use a function pointer stored in > the scanner structure to do direct SCSI or SCSI over USB. Another possibility > to distinguish between the two possibilities is the ?model? variable, also in > the scanner structure. If it is NULL it indicates a SCSI scanner, otherwise > it contains a pointer to the description of an USB scanner. A number of the > original functions had become so messed up with if-thens that I wrote > pie_usb_ counterparts to ensure that the SCSI part of the backend was not > suffering. > A note on calibration: > An outline of what happens during a scan you can find in the introductory > comment of pie_usb.h. The best reference for the commands and what they > contain is in Jan's post cited above. The scanners use two custom SCSI > commands for reading and sending calibration records. During full calibration > a record is read from the scanner, then a few calibration lines are read and > last, altered calibration data is sent back to the scanner. Although Jan and > I could figure out the meaning of most record fields, the snoops did not tell > us what was happening between reading and sending in the Windows software. > Reverse engineering seemed hopeless. I know Jan would like to have an exact > calibration procedure derived from data sheets. There is one huge custom made > chip in PIE scanners which presumably does the essential things, and of > course there is no documentation. So I started a life-science approach by > empirically determining a mathematical model and its parameters. Once one > knows what to send for exposure time, gain and offset, one can try different > values, scan an empty slide and measure mean color values of the pictures. It > melts down to find a relation between exposure time and gain and resulting > brightnesses for R, G, B because the Windows programs never changed offset or > infrared values between calibration reading and writing. In OpenOffice Calc > one can fit a number of curves to the time, gain an illumination values to > get a first idea or one can compare the measured values to ones of an own > function. In the end the functions giving the best fit were highly non-linear > and their parameters had to be determined with help of OpenOffice's > evolutionary solver. For a short description of how calibration is done look > at the comments before the pie_usb_cali functions in pie.c. If that is not > sufficient look at the pie-usb-calibration-formulas document at > http://wwwuser.gwdg.de/~mrickma/sane-proscan-7200/status-210812/pie_usb-calibration.odt > . By the way, the capability to override calibration values with values read > from text files is still in the backend, just look for pie_usb_poke in pie.c. > Shading correction was easy because the scanner tells in the calibration read > record what brightness he was aiming at (Jan's Auto-tuned full-scale-level > values) for the calibration lines. So, as one knows the outcome one can > correct. Calibration lines are always read at full resolution, i.e. every > sensor element is used. For a real scan at lower resolution the block of the > COPY (0x18) command tells which sensor elements were used so that one can > correct each pixel. > > I do not have to elaborate the dirt cleaning by infrared here because the > pie_usb_sw_post routine in pie.c follows the outline already given in > http://lists.alioth.debian.org/pipermail/sane-devel/2011-July/028810.html . > > Attached patch includes all this. The patch can be applied to current SANE > git 73d450d Sat Aug 18 23:00:14 2012. After a autoreconf [--force] and > configure ...[--enable-pthread] it can be build copied and used. > Although I tried to minimize the diff it remains rather unreadable for me. > Therefore I used doxygen style comments for the pie USB code. The html output > you find at > http://wwwuser.gwdg.de/~mrickma/sane-proscan-7200/status-210812/html . > The parent directory of that contains the patch again, the document on > calibration, the scripts which were used to process the USB snoops and the > doxygen tar ball and a screen shot using XSane. > > Now my questions: > Would code like the one in the patch be acceptable in SANE? > Would a separate backend be preferable to patching pie? > What can I do about the license issue already mentioned? I think I could just > ask Emre Celebri to allow SANE's license exception for the three routines > which were adapted from his GPLed code. He seems to be still at lsus.edu . > Regards > Michael > > > -- > sane-devel mailing list: sane-devel at lists.alioth.debian.org > http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/sane-devel > Unsubscribe: Send mail with subject "unsubscribe your_password" > to sane-devel-request at lists.alioth.debian.org -- "The truth is an offense, but not a sin"
