Hi Anu Thank you again for sharing the image. It is a challenging one and the conversation is getting pretty interesting. If you have time you still might want to post on Image.sc to see if anyone there has ideas. In my hands, generalized AI (like CellPose or Sam (Segment Anything)) did not produce a good result in '1 click' mode. However it would be fascinating to see if others can do better with the newer AI tools, perhaps by tuning parameters. Like Herbie I also encourage you to update here as well if you have time. It would be very interesting to see where you get with this project.
Hi Herbie Yes, the image I attached was a screen shot with a transparent overlay. Starting from Curtis's segmentation result with 85% of the petals identified, but a lot of clutter and 3 incorrectly split petals, it took about 30 minutes to come up with the better result which was obtained with custom programming of obtuse rules to filter Curtis's labels. I'd estimate it would take about 20 minutes to get a 'self-prediction' using a pixel based machine learning plugin like labkit (for someone already familiar with the technique). However what would potentially be much more time consuming is the process of validating on unseen images and curating additional training data if needed. So at least in my hands the time frame to get one good image partially 'automatically' using machine learning trained on the difficult parts of a single image was perhaps an hour. I'd say it took me 20 minutes to label the image manually. All these are pretty rough estimates. Anu and Herbie: I wanted to attach better images of the processing steps but the listserv would not accept the 5 images I attached. However I attached the final result and below are a description of the steps. (Like the machine-learning step, this workflow is just a (likely overfit) self-prediction, steps were tuned to the same image they were tested on). 1. Start with the pixel probability map Curtis generated with labkit. 2. Apply Li auto-threshold to get outlines of petals. 3. Filter small noise objects in outline image. 4. Invert outlines so petals are foreground. 5. Filter out small and irregular objects (area < 200, eccentricity above 0.7) (filter objects that aren't petals) 6. Merge split petals based on angle with respect to center. On Sat, Mar 9, 2024 at 1:34 PM Herbie <[email protected]> wrote: > Dear Brian, > > obviously we have only a single sample image and your segmentation of it > looks promising, although I don't understand the patchy coloring of the > areas and the lower spatial resolution of the result (is the coloring an > overlay showing some degree of transparency?) and why it is a screen-shot. > > It would be interesting to know how long it took to optimize the process > in order to come up with this result *without* any manual intervention. > > Thanks so far > > Herbie > > ::::::::::::::::::::::::::::::::::::::::::: > Am 09.03.24 um 18:09 schrieb Brian Northan: > > Hi Herbie and Anu > > > > "If "Curtis's result is almost there", why then don't we see the result?" > > > > This is a good point. If 'almost' there why don't I just show you > > the rest? Curtis's result looks to have 21 of the 24 petals segmented, > > with 3 incorrectly split in 2. So in terms of success rate one could > > argue it is over 85% there. > > > > Two strategies could get us the rest of the way there. 1) More labelling > > at the machine learning step and/or using more advanced machine learning > > (deep learning) or 2) coming up with a rule set for filter labelling and > > merging. > > > > I didn't feel comfortable with the first without having more test images > > (if I label too much on one image I'm just manually labeling that image > > anyway). So I chose the second for now. > > > > The rule set that seems to work for one image (but probably would not > > work for others) is to eliminate small labels, eliminate labels touching > > the edge, then identify the central circle (using circularity or > > eccentricity). Then for each remaining object calculate the angle and > > distance with respect to the center. If two objects have an angle that > > is close, merge them (these will be the split petals). > > > > Attached is the result. I coded it in Python, but someone good with > > ImageJ object manipulation could probably do something similar in > > ImageJ. I get the 24 petals but I make no claims this is a good method, > > it is very fragile and overfit to this one image. To automate this one > > image, I put in much more work then labelling it myself (and the labels > > are noisier). The main use of such a result is a hint that maybe > > automating a larger set is possible (or maybe not, again it will depend > > on the variability of the larger set). > > > > Brian > > > > On Sat, Mar 9, 2024 at 8:13 AM Herbie <[email protected] > > <mailto:[email protected]>> wrote: > > > > Dear Brian, > > > > I very much like this... > > > > "However, assessing the potential of automation on this problem is > > still > > valuable,perhaps you will have to do another batch of images in a few > > months or something, and the insights may help others facing similar > > problems." > > > > ...social aspect but with my today's academic experience I'm tempted > to > > call it an utopian expectation. > > > > It would be interesting to know which status people on Lists and Fora > > have who *continuously* provide substantial and unselfish help and > > assistance. > > > > Regards > > > > Herbie > > > > ::::::::::::::::::::::::::::::::::::::::::: > > Am 09.03.24 um 13:03 schrieb Brian Northan: > > > Hi Anu > > > > > > There are many similar problems on the Image.sc message board, > > and both > > > classical and advanced AI methods are often shown to be good at > > solving the > > > one image the researcher shares with the community. What is > > often lacking > > > is feedback on how the solution(s) suggested, work on the entire > > set of > > > images the researcher needs to process. > > > > > > I really like Curtis's approach. However the question is how > > will this > > > work on the other 50 or more images? It will depend on the > > variation in > > > the image set. As Herbie points out, we have no idea how > > Curtis's method > > > (or other solution) will work on the entire set. > > > > > > I hesitate to ask researchers to share more data, as I realize > > you are > > > busy, and you may be constrained as to how much (possibly > > unpublished) data > > > you are able to release publicly. > > > > > > However if it is at all possible to share more data (5-10 > > examples) it > > > would really help in assessing whether proposed solutions > > generalize to > > > your entire image set. > > > > > > At this point I agree that it may be faster to manually do it. > > However, > > > assessing the potential of automation on this problem is still > > valuable, > > > perhaps you will have to do another batch of images in a few > > months or > > > something, and the insights may help others facing similar > problems. > > > > > > Brian > > > > > > On Sat, Mar 9, 2024 at 6:11 AM Herbie <[email protected] > > <mailto:[email protected]>> wrote: > > > > > >> Greetings Anu, > > >> > > >> because you called my suggestion as being your "last option" I > > should > > >> like to remark that this idea is exactly *contrary* to what I've > > suggested. > > >> > > >> My prognosis is, that you will invest a lot of time with > > learning and > > >> trying various (advanced) approaches and then realize that none > > of them > > >> works fully automatically, i.e. will need additional manual > > >> interventions that again take time. In the end you may realize > > that the > > >> better (quicker) approach would have been to start immediately > > with the > > >> manual segmentation of your 50 images. > > >> > > >> Last but not least, if things would be so easy and economic with > > using > > >> (advanced) approaches, why then don't we see any results that > > are fully > > >> automatically obtained from your sample image? > > >> > > >> Good luck > > >> > > >> Herbie > > >> > > >> :::::::::::::::::::::::::::::::::::::::: > > >> Am 08.03.24 um 19:01 schrieb anusuya pal: > > >>> Thanks so much everyone for suggesting so many ways. Thanks, > > Herbie, yes, > > >>> that's the last option I thought. :-) > > >>> > > >>> I really like Ankit's proposal as it's very much automated. The > > idea > > >> given > > >>> by Michael -- I am kind of doing that for finding the spacing > > between the > > >>> consecutive petal like patterns. But, that doesn't give me a > good > > >> estimate > > >>> for all of my images, as it is just one type of pattern. > > >>> > > >>> I also like Curtis's idea, I need to play with that as > > suggested for the > > >>> various patterns to see which one works the best. > > >>> > > >>> I really appreciate your valuable time and suggestions. > > >>> > > >>> Thanks, > > >>> Anu > > >>> > > >>> On Saturday, March 9, 2024, Curtis Rueden > > <[email protected] <mailto:[email protected]>> > > >> wrote: > > >>> > > >>>> Hi Anu, > > >>>> > > >>>> I think your segmentation can be automated, but it is a bit > > tricky. > > >> Here is > > >>>> a quick attempt I made: > > >>>> > > >>>> 1. Labkit - https://imagej.net/plugins/labkit/ > > <https://imagej.net/plugins/labkit/> > > >>>> This is a machine-learning based pixel classification, where > > you do > > >> manual > > >>>> painting over the different areas of your image. Then train > > it, and > > >> paint > > >>>> again over the parts it got wrong. Repeat until it learns well > how > > >> things > > >>>> should be. > > >>>> > > >>>> Here is how it looked for me after I did this process back and > > forth a > > >>>> couple of times: > > >>>> > > >>>> [image: labkit-small.png] > > >>>> > > >>>> As you can see, it is not perfect, but it gets close enough > > that you can > > >>>> then do additional steps afterward to extract the information > > you want. > > >>>> Then, you can save the classifier and apply it to as many > > other similar > > >>>> images as you want. > > >>>> > > >>>> Note that Labkit (at least in my hands today) has an annoying > > bug where > > >>>> after running the classifier (Ctrl+Shift+T), the pencil tool > > sometimes > > >>>> stops being able to paint lines until you click (or Alt+Tab) > > away from > > >> the > > >>>> Labkit window and then back. > > >>>> > > >>>> 2. Export probability map to ImageJ > > >>>> > > >>>> This gets you back to a regular image window, which you can > then > > >> manipulate > > >>>> with other plugins. > > >>>> > > >>>> You might always want to save this image to a TIFF file now, > > since it > > >> will > > >>>> serve as a good starting point for further experimentation. > > >>>> > > >>>> 3. Smooth the image to reduce noise. I used the Kuwahara > > filter. But it > > >>>> didn't want to run on a 32-bit multichannel image, so I had to > > first run > > >>>> Image > Type > 8-bit and then Duplicate only the first slice > > of the > > >> image. > > >>>> > > >>>> The easiest way to run it is to type "kuwa" into the search > > bar of Fiji. > > >>>> > > >>>> After running this filter with a smoothing window of 5, my > > image looked > > >>>> like this: > > >>>> > > >>>> [image: smoothed-small.png] > > >>>> > > >>>> 4. Do the actual segmentation with the Morphological > Segmentation > > >> plugin, > > >>>> part of MorphoLibJ. https://imagej.net/plugins/morpholibj > > <https://imagej.net/plugins/morpholibj> > > >>>> > > >>>> For this plugin you will need to enable the IJPB-plugins > > update site via > > >>>> Help > Update..., "Manage Update Sites" button, in Fiji. > > >>>> > > >>>> I left the input image as Border Image, changed Tolerance to > > 30, clicked > > >>>> Run, and then changed the Results Display to "Catchment > > basins". Here is > > >>>> what that looked like: > > >>>> > > >>>> [image: morpholibj-small.png] > > >>>> > > >>>> As you can see, it erroneously bisected two of the regions on > > the bottom > > >>>> half, as well as one on the top half, but it got most of then > > right. > > >>>> > > >>>> 5. You could then click "Create image" to make another image > > and measure > > >>>> the number of pixels of each color to get the size of each > > region. And > > >>>> filter out results that aren't close to the expected size, or > > aren't at > > >> the > > >>>> correct (X,Y) coordinates to be one of the petal shapes. > > >>>> > > >>>> I would also suggest to give CellPose a try—I did not try it, > > but it > > >> does > > >>>> very well on a wide variety of input images. > > >>>> > > >>>> You might get better answers on https://forum.image.sc > > <https://forum.image.sc> rather than > > >> here, > > >>>> since the state-of-the-art for segmenting scientific images > > has changed > > >> a > > >>>> lot in recent years and there are many more powerful tools than > > >> classical > > >>>> ImageJ-based segmentation now. > > >>>> > > >>>> Regards, > > >>>> Curtis > > >>>> > > >>>> -- > > >>>> Curtis Rueden > > >>>> Software architect, LOCI/Eliceiri lab - > > https://uw-loci.github.io/ <https://uw-loci.github.io/> > > >>>> ImageJ2 lead, Fiji maintainer - > > https://imagej.net/people/ctrueden < > https://imagej.net/people/ctrueden> > > >>>> Have you tried the Image.sc Forum? https://forum.image.sc/ > > <https://forum.image.sc/> > > >>>> > > >>>> > > >>>> On Fri, Mar 8, 2024 at 1:48 AM anusuya pal > > <[email protected] <mailto:[email protected]>> > > >> wrote: > > >>>> > > >>>>> Dear all, > > >>>>> > > >>>>> I want to measure the area of the flower-like patterns as > > shown in the > > >>>>> image. I can do it manually, but I have more than 50 images. > > Do you > > >> have > > >>>>> any suggestions for doing it automatically? > > >>>>> > > >>>>> Thanks for your help, > > >>>>> Anu > > >>>>> > > >>>>> -- > > >>>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > >>>>> > > >>>> > > >>>> -- > > >>>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > >>>> > > >>> > > >>> -- > > >>> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > >>> > > >> > > >> -- > > >> ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > >> > > > > > > -- > > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > > > > > > -- > > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > > <http://imagej.nih.gov/ij/list.html> > > > > -- > ImageJ mailing list: http://imagej.nih.gov/ij/list.html > -- ImageJ mailing list: http://imagej.nih.gov/ij/list.html
