Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Joe, You are happy, = we are happy. Glad to see that tracker's package fits your use-case. That's why we make software: to create symbioses between available software and companies. To see it being used by uses like yours. Report back. Regards, Philip On Thu, 2016-01-21 at 18:23 -0500, Joe

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hey Joe, On Thu, Jan 21, 2016 at 3:38 AM, Joe Rhodes wrote: > Indeed. I was mistaken about the memory usage. Tracker-miner-fs has now > stabilized at about 265 MB of RAM. I force a re-crawl once every two hours > to update the index. Thanks for the update! would be

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Carlos: Here are the numbers as requested: tracker status -a | grep -e "nfo:Folder" -e "nfo:FileDataObject" nfo:FileDataObject = 1704170 nfo:Folder = 286147 As for the Netatlak deal, I was working with the developer. It seemed that if you ran a query from a Mac that would return too many

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

On Thu, Jan 21, 2016 at 07:07:17AM -0500, Joe Rhodes wrote: > Carlos: > > Here are the numbers as requested: > > tracker status -a | grep -e "nfo:Folder" -e "nfo:FileDataObject" > nfo:FileDataObject = 1704170 > nfo:Folder = 286147 > > > As for the Netatlak deal, I was working with the

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

On Thu, 2016-01-21 at 07:07 -0500, Joe Rhodes wrote: > Carlos: > > Here are the numbers as requested: > > tracker status -a | grep -e "nfo:Folder" -e "nfo:FileDataObject" > nfo:FileDataObject = 1704170 > nfo:Folder = 286147 About 300 MB VmRSS for 1,704,170 and 286,147 directories. That's

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

On Thu, 2016-01-21 at 14:02 +0100, Carlos Garnacho wrote: > > Yeah, afair the problem with Apples Spotlight RPC protocol is that it > > has no way returning results in chunks and tell the client to come > > back immediately for more results. > > > > Apples Spotlight RPC server simply puts *all*

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Philip: tracker-store is sitting at about 38 MB of RAM. So over all, I'm pretty happy. Again, keep in mind that I've turned off all the full-text searching. So this is just an index of file names. Cheers! -Joe > On Jan 21, 2016, at 7:14 AM, Philip Van Hoof wrote: >

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Indeed. I was mistaken about the memory usage. Tracker-miner-fs has now stabilized at about 265 MB of RAM. I force a re-crawl once every two hours to update the index. I've disabled the content searching for now by removing all the rules. Nothing to do with Tracker. It just took too long

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, On Mon, Jan 18, 2016 at 2:02 AM, Joe Rhodes wrote: > Carlos: > > I just tried the 1.7.2 tarball. I think we may still have a leak. The two > processes were heading up over 600 MB's. > > Here are the valgrind outputs. I hope this helps: > >

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

On Mon, 2016-01-18 at 12:57 +0100, Carlos Garnacho wrote: > As expected, the valgrind logs are a lot more favorable. > Tracker-extract reports: > > ==21759==definitely lost: 168 bytes in 1 blocks > ==21759==indirectly lost: 6,046 bytes in 2 blocks > > And those seem to be a memory pool

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, Philip, On Wed, Jan 13, 2016 at 8:55 AM, Philip Van Hoof wrote: > Hi Joe, > > Ok, that's fine. It would have been nice if we could let you do a > valgrind re-run of the same use-case to see if we catch all the memory > leaks before Carlos makes a new release. > >

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, Ok, that's fine. It would have been nice if we could let you do a valgrind re-run of the same use-case to see if we catch all the memory leaks before Carlos makes a new release. But this way we'll do a post-release check of your use-case. If we see more memory leaks after your retry we

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, If you run a debian based distro: apt-get build-dep tracker. Else in this case you need to install a package which is usually called gobject-introspection. ps. for the office files you'll need GFS, libgfs-1-dev or something. Kind regards, Philip On Mon, 2016-01-11 at 21:33 -0500, Joe

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Carlos, Looks like my git-account has been closed on GNOME, so here is a patch for one of the issues in that valgrind. Kind regards, Philip On Sun, 2016-01-10 at 16:05 -0500, Joe Rhodes wrote: > Carlos: > > Yes, there are a LOT of files on this volume. The makeup of the 5 TB of data >

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Philip :) On Mon, Jan 11, 2016 at 11:21 AM, Philip Van Hoof wrote: > Hi Carlos, > > Looks like my git-account has been closed on GNOME, so here is a patch > for one of the issues in that valgrind. Thanks! The patch is now in master and previous stable branches. And I

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Carlos, et. al., I'm sorry, but I cannot seem to build the master branch right now. I ran the autogen.sh script and then configure dies on me with this: checking for pkg-config... /usr/bin/pkg-config checking pkg-config is at least version 0.16... yes ./configure: line 19136: syntax error near

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, On Sun, Jan 10, 2016 at 10:05 PM, Joe Rhodes wrote: > Carlos: > > Yes, there are a LOT of files on this volume. The makeup of the 5 TB of data > is PDFs, Photoshop files, Word docs, InDesign & Illustrator docs. There are > very few large files like MP3's or

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

On Mon, 2016-01-11 at 11:51 +0100, Carlos Garnacho wrote: Hi Carlos, > > > > Looks like my git-account has been closed on GNOME, so here is a patch > > for one of the issues in that valgrind. > > Thanks! The patch is now in master and previous stable branches. Ok, thanks for pushing it for me.

[Tracker] Memory usage for tracker-extract and tracker-miner-fs

I have just compiled and installed tracker-1.7.1 on a CentOS 7.1 box. I just used the default configuration ("./configure" with no additional options). I'm indexing around 5 TB of data. I'm noticing that both the tracker-extract and tracker-miner-fs processes are using a large amount of

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Carlos: Yes, there are a LOT of files on this volume. The makeup of the 5 TB of data is PDFs, Photoshop files, Word docs, InDesign & Illustrator docs. There are very few large files like MP3's or videos. If I disable all the extractors and just build an index based on file names, I get an

Re: [Tracker] Memory usage for tracker-extract and tracker-miner-fs

Hi Joe, On Sun, Jan 10, 2016 at 6:40 PM, Joe Rhodes wrote: > I have just compiled and installed tracker-1.7.1 on a CentOS 7.1 box. I > just used the default configuration ("./configure" with no additional > options). I'm indexing around 5 TB of data. I'm noticing that