Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

It's certainly a good idea.

Writing a Spotlight plug-in is extremely straight forward - just fire  
up Xcode 2 and create a new "Metadata Importer" project. You just  
have one function that you need to provide:

Boolean GetMetadataForFile(void* thisInterface,
CFMutableDictionaryRef attributes,
CFStringRef contentTypeUTI,
CFStringRef pathToFile)

Don't forget to modify the Info.plist and possibly the schema.xml file.

Also - you'll probably need to create an installer for the importer  
so that when installed it forces a reindex, see: http://www.apple.com/ 
downloads/macosx/submit/installers.html

Now I believe that Spotlight is smart enough to detect when a file is  
created/written that needs importing so you wont have to manually  
force any re-indexing yourself.

I'm currently writing a spotlight importers for the GPX xml file  
format and for another inhouse project of mine.

 Jon.

On May 10, 2005, at 11:28, David Reed wrote:

> I noticed Spotlight doesn't index Python files. I'm not certain how
> useful it would be but I was thinking about trying to get spotlight
> to index the name of classes and functions/methods (i.e., you could
> enter a class name and spotlight would find the Python file
> containing the class). I don't care if it does it on the fly as files
> are updated, but thought it would be nice to be able to run a Python
> script that would index all Python files in a directory. You could
> use the inspect module to grab the class/function/method names and
> tell spotlight to include them. I know there are some command line
> tools to interact with spotlight, but after a quick look at them,
> nothing jumped out at me for doing this.
>
> Is anyone interested in this (i.e., would you find it useful) and if
> so, does anyone understand Spotlight well enough to point me in the
> right direction (pointers to specific documentation are welcome).
>
> Thanks,
> Dave
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

On May 10, 2005, at 14:49, David Reed wrote:

> I've looked at the example at:
>
> http://developer.apple.com/documentation/Carbon/Conceptual/ 
> MDImporters/Concepts/WritingAnImp.html#//apple_ref/doc/uid/ 
> TP40001275-CJBEJBHH
>
> I don't yet understand what NSDictionary tempDict contains after it  
> "loads the document at the specified location".
> My thought would be to read the file and look for def and class and  
> then grab the next word after it and add it to the index. Looks  
> like kMDItemTextContent is what could be used to do that.
>
> I could read the file in C and look for def/class and then would  
> just need to figure out how to use the kMDItemTextContent class.
>
> Am I on the correct track?

See:

http://developer.apple.com/documentation/Carbon/Reference/MDItemRef/ 
Reference/chapter_1.3_section_2.html#//apple_ref/doc/c_ref/ 
kMDItemTextContent

The best way I think would be to create new spotlight metadata  
attribute types for "Class Name" and "Function Name" (or see if C/ 
ObjC files already provide similar useful keys (just checked - they  
don't seem to). Then find all the class/function names in the file  
and add them to the new attributes. Then you'd join all this data  
together with whitespace and put it into the kMDItemTextContent  
attribute.

I'm not sure how useful it is to index function & class names though.  
There could be a heck of a lot of those - most of which might not be  
too useful. Wouldn't doing something with the Python __doc__ blocks  
be more useful?

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

I'll have a play with it later today. I don't think it should be too  
difficult at all.

 Jon.

On May 10, 2005, at 16:22, David Reed wrote:


> I don't think I'd want all variables/symbols to be indexed, but I
> think function and class names might be nice and adding the
> documentation strings might be worthwhile too. It appears I still
> need to learn more about Objective-C although one of Jonathan's
> emails makes me think I could just read the file and make a space
> separated string of the functions and class names and that would
> about take care of it.
>
> If someone who knows more about Objective-C wants to take a stab at
> this feel free as I probably won't get to learning more about
> Objective-C for a couple more months.
>

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

I've made a first pass at it and have a Spotlight importer that calls  
a built-in Python function to import a file's metadata.

I started to look at module inspect to find out how to extract  
information from a Python module but then realised that I'd need to  
import the file the importer is analysing. This would mean it will be  
executing arbitrary code inside that file. That's got to be a bad  
thing for security reasons.

So instead I'm just going to have to use string processing to scan  
the file instead. Are there any modules out there for extracting  
information from Python script files?

 Jon.

On May 10, 2005, at 16:58, Jonathan Wight wrote:






> I'll have a play with it later today. I don't think it should be too
> difficult at all.
>
>  Jon.
>
> On May 10, 2005, at 16:22, David Reed wrote:
>
>
>
>
>
>
>
>
>> I don't think I'd want all variables/symbols to be indexed, but I
>> think function and class names might be nice and adding the
>> documentation strings might be worthwhile too. It appears I still
>> need to learn more about Objective-C although one of Jonathan's
>> emails makes me think I could just read the file and make a space
>> separated string of the functions and class names and that would
>> about take care of it.
>>
>> If someone who knows more about Objective-C wants to take a stab at
>> this feel free as I probably won't get to learning more about
>> Objective-C for a couple more months.
>>
>>
>>
>>





___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

I looked at parser but discounted it too soon. After a bit of playing  
I managed to get the following code to provide lists of class names  
and function names.

Thanks Bob!

 Jon.

#!/usr/bin/python

import parser
import symbol

def find(inTuple, inSymbol):
 theClasses = []
 if inTuple[0] == inSymbol:
 print 'FOUND'
 theClasses += [ inTuple[2][1] ]
 for theItem in inTuple[1:]:
 if type(theItem) == type(()):
 theClasses += find(theItem, inSymbol)
 return theClasses

theSource = open('/Users/schwa/Desktop/Test.py').read()
ast = parser.suite(theSource)
tup = ast.totuple()

print find(tup, symbol.classdef)
print find(tup, symbol.funcdef)



On May 10, 2005, at 21:52, Bob Ippolito wrote:

>
> On May 10, 2005, at 8:52 PM, Jonathan Wight wrote:
>
>
>> I've made a first pass at it and have a Spotlight importer that calls
>> a built-in Python function to import a file's metadata.
>>
>> I started to look at module inspect to find out how to extract
>> information from a Python module but then realised that I'd need to
>> import the file the importer is analysing. This would mean it will be
>> executing arbitrary code inside that file. That's got to be a bad
>> thing for security reasons.
>>
>> So instead I'm just going to have to use string processing to scan
>> the file instead. Are there any modules out there for extracting
>> information from Python script files?
>>
>
> You want to use the parser module.
>
> -bob
>
>

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

OK. The first version is online at:

http://toxicsoftware.com/_private/Python%20Metadata%20Importer.pkg.zip

The installer installs the mdimporter to /Library/Spotlight/ and then  
calls mdimport -r to force Spotlight to reindex all Python files.

You _may_ need Xcode installed to have this work successfully (Apple  
defines the "public.python-script" UTI type - and it _may_ be defined  
in Xcode - I need to check).

This version scans through .py files looking for function names and  
class names. It adds them to the file's 'org_python_functions' and  
'org_python_classes' metadata attributes respectively

You can view the python specific metadata using mdls

[EMAIL PROTECTED] Python Metadata Importer$ mdls *.py
myimporter.py -
kMDItemContentType = "public.python-script"
... other metadata not shown ...
kMDItemKind= "Python Script"
org_python_functions   = (find, main)

And find it using mdfind:

[EMAIL PROTECTED] Spotlight$ mdfind "org_python_functions == 'main'" |  
wc -l
   41

You can also use the Finder to search for functions/classes: bring up  
a new search window, then choose "other…" in the attribute menu, in  
the attribute search window look for the keyword "Python"... It is  
quite impressive doing a search for python function names from the  
finder.

The importer actually uses an embedded python interpreter to parse  
the python files (thanks to Bob for suggest the parser module). You  
can see the python file inside the Resources folder of the importer.  
The code currently leaks a few PyObjects so I'll be working on that  
in the next version.

Currently I do not add anything to kMDItemTextContent or any other  
metadata attribute but this kind of stuff should be easy to do (just  
a one line change to the Python code).

Any feedback quite welcome.

Cheers.

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

It is entirely possible I screwed something up and that it doesn't  
work on anything other than my Powerbook ;-)

However:

As the last part of the install process the installer kicks off a  
script to reindex the Python files on the hard drive. This could be  
failing. To manually reindex runt this:

mdimport -r /Library/Spotlight/Python\ Metadata\ Importer.mdimporter/

You should see a response like this:

2005-05-11 10:55:48.004 mdimport[21563] Asking server to reimport  
files with UTIs: (
 "dyn.ah62d4rv4ge81a8k",
 "dyn.ah62d4rv4gq81k3p2su11upputf4gu55sfz30g6xmsb4a",
 "public.python-script"
)

On my machine with 2748 python files indexable by Spotlight ( mdfind  
'kMDItemContentType == "public.python-script' | wc -l ) it seems to  
take less than 5 minutes with 2 mdimport tasks running at between  
30-70% CPU.

Note that spotlight doesn't index Python.framework by default - so  
you'd need to configure that somehow (perhaps via the .Spotlight-V100  
directory).

To confirm that the Python mdimporter is working run mdimport against  
a python file (with classes & functions declared) with debugging  
turned on. You should see something like this, note that mdimport is  
using the Python mdimporter and that the 'org_python_functions'  
attribute is present in the output:

mdimport -d 2 myimporter.py

2005-05-11 11:03:51.318 mdimport[21650] Import '/Volumes/Home/Users/ 
schwa/Desktop/Python Metadata Importer/myimporter.py' type  
'public.python-script' using 'file://localhost/Library/Spotlight/ 
Python%20Metadata%20Importer.mdimporter/'
2005-05-11 11:03:51.622 mdimport[21650] Done.
2005-05-11 11:03:51.624 mdimport[21650] Sending attributes of '/ 
Volumes/Home/Users/schwa/Desktop/Python Metadata Importer/ 
myimporter.py' to server.  Attributes: '{
 "com_apple_metadata_modtime" = 137481386;
 kMDItemContentCreationDate = "2005-05-11 00:03:18 -0400";
 kMDItemContentModificationDate = "2005-05-11 01:16:26 -0400";
 kMDItemContentType = "public.python-script";
 kMDItemContentTypeTree = (
 "public.python-script",
 "public.shell-script",
 "public.script",
 "public.source-code",
 "public.plain-text",
 "public.text",
 "public.data",
 "public.item",
 "public.content"
 );
 kMDItemDisplayName = {"" = "myimporter.py"; };
 kMDItemKind = {"" = "Python Script"; };
 "org_python_classes" = ();
 "org_python_functions" = (find, main);

Let me know if this works.

 Jon.

On May 11, 2005, at 10:51, David Reed wrote:

> Thanks for working on this. I'm not certain it is working  
> correctly. I installed it about 3 hours ago so I would think it has  
> had time to index my drive by now. It does seem to have indexed  
> files that are part of python packages but it hasn't index Python  
> files in my home directory and subdirectories. Is it skipping these  
> on purpose or does it somehow not realize they are Python files  
> (most of them were created with emacs and many on Linux systems  
> before I switched to a Mac)? They all do have a .py extension.
>
> Also, how do you uninstall your plugin?
>
> I do have XCode installed.

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

On May 11, 2005, at 11:16, Schollnick, Benjamin wrote:

>>> had time to index my drive by now. It does seem to have indexed
>>> files that are part of python packages but it hasn't index Python
>>> files in my home directory and subdirectories. Is it
>>>
>> skipping these
>>
>>> on purpose or does it somehow not realize they are Python files
>>> (most of them were created with emacs and many on Linux systems
>>> before I switched to a Mac)? They all do have a .py extension.
>>>
>
> So, to summarize...
>
> 1) It has indexed files that are part of Python packages
> 2) It has not indexed files in his home directory
> a) Most were created with Emacs / Linux systems
>
> So the UTI for python files may not be attached to those files
> MetaData
>
> Could that be preventing the indexer from operating?

I'd be surprised. The python UTI declaration is in the system (not in  
Xcode as I originally believed) and it looks like this:

 
 UTTypeConformsTo
 
 public.shell-script
 
 UTTypeDescription
 Python script
 UTTypeIdentifier
 public.python-script
 UTTypeTagSpecification
 
 public.filename-extension
 
 py
 
 public.mime-type
 
 text/x-python-script
 
 
 

So basically anything with a .py extension should be indexed.

Use mdls and mdfind to find out files are not being indexed.

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

On May 11, 2005, at 10:51, David Reed wrote:

> Also, how do you uninstall your plugin?

Remove "Python Metadata Importer.mdimporter" from /Library/Spotlight
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

For me it worked:

 "org_python_functions" = ("get_bits", "set_lsb", "get_lsb",  
"value_from_bits");

But then I just made a new BBEdit Bits.py file and pasted your code  
into it.

The importer does seem to be crashing, but I'm not sure whether the  
python or the ObjC is dying. Is there a crash log in either ~/Library/ 
Logs/Crash Logs or /Library/Logs/Crash Logs.

I'm working on version 0.2 with slightly better error handling and a  
lot less leaking of PyObject pointers.

 Jon.

On May 11, 2005, at 13:21, David Reed wrote:

> It doesn't appear to be working:
>
> 510 mac:~/src/CS160/ImageMessage $ mdimport -d 2 bits.py
> 2005-05-11 13:17:48.498 mdimport[3670] Import '/Users/dreed/src/ 
> CS160/ImageMessage/bits.py' type 'public.python-script' using  
> 'file://localhost/Library/Spotlight/Python%20Metadata% 
> 20Importer.mdimporter/'
> 2005-05-11 13:17:48.879 mdimport[3670] -[FileProcessor  
> importMetadataFromFileAtPì¤í¸ íìot exception Conversion to encoding  
> 30 failed for string "ì
> 2005-05-11 13:17:48.897 mdimport[3670] Import '/Users/dreed/src/ 
> CS160/ImageMessage/bits.py' type 'public.python-script' no mdimporter
> 2005-05-11 13:17:48.900 mdimport[3670] Sending attributes of '/ 
> Users/dreed/src/CS160/ImageMessage/bits.py' to server.  Attributes: '{
> "_kMDItemImporterCrashed" = 1;
> "com_apple_metadata_modtime" = 137504114;
> kMDItemContentCreationDate = 2005-05-11 07:35:14 -0400;
> kMDItemContentModificationDate = 2005-05-11 07:35:14 -0400;
> kMDItemContentType = "public.python-script";
> kMDItemContentTypeTree = (
> "public.python-script",
> "public.shell-script",
> "public.script",
> "public.source-code",
> "public.plain-text",
> "public.text",
> "public.data",
> "public.item",
> "public.content"
> );
> kMDItemDisplayName = {"" = "bits.py"; };
> kMDItemKind = {
> "" = PlainTextType;
> ca = "Fitxer de Text Planer";
> da = "Almindeligt tekstdokument";
> de = "Reine Text-Datei";
> en = "Plain Text File";
> fr = "Fichier texte brut";
> ja = "\U6a19\U6e96\U30c6\U30ad\U30b9\U30c8\U66f8\U985e";
> ko = "\Uc77c\Ubc18 \Ud14d\Uc2a4\Ud2b8 \Ud30c\Uc77c";
> ru = "\U041f\U0440\U043e\U0441\U0442\U043e\U0439 \U0442 
> \U0435\U043a\U0441\U0442";
> "zh-Hant" = "\U7d14\U6587\U5b57\U6a94";
> };
> }'
>
>
> The bits.py file is just:
>
> #!/usr/bin/env python
>
> #- 
> -
> # bits.py
> # Dave Reed
> # 05/09/2005
> #- 
> -
>
> def get_bits(value, n=8):
>
> '''get_bits(value, n):
>
> generator to return individual bits in a value of n bits'''
>
> pos_value = 2 ** (n-1)
>
> # if a single character string, convert to ASCII code
> try:
> value = ord(value)
> except:
> pass
>
> # for each bit
> for i in range(n):
> # get bit n
> bit = value & pos_value
> # shift so we can get next bit
> value = value << 1
> # if bit is pos_value, that bit was 1
> if bit == pos_value:
> yield 1
> else:
> yield 0
>
> #- 
> -
>
> def set_lsb(x, b):
>
> '''set_lsb(x, b):
>
> returns x with lsb of x set to b (b is 0 or 1)'''
>
> return (x & 254) | b
>
> #- 
> -
>
> def get_lsb(x):
>
> '''get_lsb(x):
>
> returns lsb of x (0 or 1)'''
>
> return x & 1
>
> #- 
> -
>
> def value_from_bits(bits):
>
> '''value_from_bits(bits):
>
> takes a sequence of bit values (each item is 0 or 1) and  
> returns the
> integer corresponding to that sequence'''
>
> value = 0
> for b in bits:
> value = 2 * value + b
> return value
>
> #- 
> -
>
>
> Dave
>
>
>

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] [Pythonmac-SIG] Spotlight and Python

[Jonathan Wight]

Version 0.7 is now online at:

http://toxicsoftware.com/_private/Python%20Metadata%20Importer/Version 
%200.7/Python%20Metadata%20Importer.dmg

Thanks to David Reed for helping me test and iron out the Unicode  
problems.

This version should work for pretty much anyone (famous last words).  
I've added added support for Unicode data, NSDate support, more  
robust error handling.

I'm going to add support for file level PythonDoc comments and for  
the file level python metadata (__author__, __date__ and so on) in  
the next version. I'll be putting the code online soon.

 Jon.

On May 11, 2005, at 02:28, Jonathan Wight wrote:



> OK. The first version is online at:
>
> http://toxicsoftware.com/_private/Python%20Metadata%20Importer.pkg.zip
>
> The installer installs the mdimporter to /Library/Spotlight/ and then
> calls mdimport -r to force Spotlight to reindex all Python files.
>
> You _may_ need Xcode installed to have this work successfully (Apple
> defines the "public.python-script" UTI type - and it _may_ be defined
> in Xcode - I need to check).
>
> This version scans through .py files looking for function names and
> class names. It adds them to the file's 'org_python_functions' and
> 'org_python_classes' metadata attributes respectively
>
> You can view the python specific metadata using mdls
>
> [EMAIL PROTECTED] Python Metadata Importer$ mdls *.py
> myimporter.py -
> kMDItemContentType = "public.python-script"
> ... other metadata not shown ...
> kMDItemKind= "Python Script"
> org_python_functions   = (find, main)
>
> And find it using mdfind:
>
> [EMAIL PROTECTED] Spotlight$ mdfind "org_python_functions == 'main'" |
> wc -l
>41
>
> You can also use the Finder to search for functions/classes: bring up
> a new search window, then choose "other…" in the attribute menu, in
> the attribute search window look for the keyword "Python"... It is
> quite impressive doing a search for python function names from the
> finder.
>
> The importer actually uses an embedded python interpreter to parse
> the python files (thanks to Bob for suggest the parser module). You
> can see the python file inside the Resources folder of the importer.
> The code currently leaks a few PyObjects so I'll be working on that
> in the next version.
>
> Currently I do not add anything to kMDItemTextContent or any other
> metadata attribute but this kind of stuff should be easy to do (just
> a one line change to the Python code).
>
> Any feedback quite welcome.
>
> Cheers.
>
>  Jon.
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig
>
>
>



___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

Version 0.9 (everything done exception final documentation tidy-up)  
is online at:

http://svn.toxicsoftware.com/svn/toxic_public/trunk/Freeware/Python% 
20Metadata%20Importer/Releases/Python%20Metadata%20Importer.dmg

Source is available from:

http://svn.toxicsoftware.com/svn/toxic_public/trunk/Freeware/Python% 
20Metadata%20Importer

Feedback MOST welcome.

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Pythonmac-SIG] ANN: Python Metadata Importer

[Jonathan Wight]

And again with long URLs fixed ;-)

I released the code under the GPL - with minor modifications to the  
code I think anyone should be able to create Spotlight Importers  
using python (without depending on PyObjC)...

-

Version 0.9 (everything done exception final documentation tidy-up)
is online at:



Source is available from:



Feedback MOST welcome.

  Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] ANN: Python Metadata Importer

[Jonathan Wight]

On May 19, 2005, at 13:32, Ronald Oussoren wrote:


> On 19-mei-2005, at 17:49, Jonathan Wight wrote:
>
>
>> And again with long URLs fixed ;-)
>>
>> I released the code under the GPL
>>
>>
> Why GPL?
>

Why not?

Suggest something different.


>> - with minor modifications to the
>> code I think anyone should be able to create Spotlight Importers
>> using python (without depending on PyObjC)...
>>
>>
>
> Is that a challenge? You cannot create importers using PyObjC at  
> the moment.
>

Not at all. But feel free to take it as challenge if you wish.  
Pistols at dawn? ;-)

I chose not to use PyObjC because:

1: I started this before the PyObjC guys had officially released  
PyObjC for 10.4

2: I'm not an expert in PyObjC.

3: I wasn't sure how to bundle PyObjC with the importer without  
conflicting with PyObjC already on their machine (see #2)

4: It was pretty quick and easy even without PyObjC.

5: It was fun. ;-)

 Jon.


___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] ANN: Python Metadata Importer

[Jonathan Wight]

On May 19, 2005, at 13:48, Bob Ippolito wrote:

>
> On May 19, 2005, at 1:32 PM, Ronald Oussoren wrote:
>
>
>> On 19-mei-2005, at 17:49, Jonathan Wight wrote:
>>
>>
>>
>>> And again with long URLs fixed ;-)
>>>
>>> I released the code under the GPL
>>>
>>>
>> Why GPL?
>>
>
> Probably because he wants someone else to re-implement it :)

Um no. I wouldn't have gone through the bother or writing it, testing  
it and then releasing it if I wanted someone else to re-implement it.

I'm moving all my free source code i write over to the GPL.

>>> - with minor modifications to the
>>> code I think anyone should be able to create Spotlight Importers
>>> using python (without depending on PyObjC)...
>>>
>>>
>>
>> Is that a challenge? You cannot create importers using PyObjC at  
>> the moment.
>>
>
> Sure you can, but you would still have to write about three lines  
> of Objective-C code beyond the template..

Go for it then.

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Free Python code and Licenses

[Jonathan Wight]

On May 20, 2005, at 03:00, Bob Ippolito wrote:


>> Does it allow the _creation_ of custom metadata tags on files? Or are
>> we restricted to the ones defined by Apple?
>>
>>
>
> Metadata importers can define whatever key/value pairs they want to.
> Only certain keys are used intelligently by Apple's Spotlight UI, but
> you can define whatever you want and there are APIs to query
> Spotlight for your own purposes.
>

One problem with defining your own keys is that there are no  
guarantees that other importers will use the same keys even if  
they're representing the same thing...

For example I'm defining two custom metadata attribute keys:  
org_python_classes & org_python_functions to represent the list of  
all classes and functions found within the source file. It would be  
good if (for example) a PHP importer used the same keys as my  
importer (perhaps with more generic key names) to represent the same  
kinds of data found in PHP files. Unfortunately there is no central  
repository (like Apples type & creator database) for Spotlight  
metadata keys so it is really hard for importer authors to cooperate.

 Jon.

___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

Just to let everyone know that my Python Metadata Importer plug-in  
was released the other day.

 http://toxicsoftware.com/blog/index.php/weblog/ 
python_metadata_importer_101_released/

I decided to change the license to BSD after Bob and co brought up  
some good points, so now anyone can use it as a basis for writing  
Python mdimporters without having to deal the GPLness.

The plug-in seems to work fine and unless anyone has some feature  
suggestions for it I hope I won't post about it any more on this  
mailing list ;-)

Quick screenshot of the Get Info window showing a .py file: http:// 
toxicsoftware.com/blog/images/uploads/PythonMetaImporterGetInfo.png

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

On May 26, 2005, at 00:38, Bob Ippolito wrote:

> On May 25, 2005, at 9:13 PM, Jonathan Wight wrote:
>
>
>> Just to let everyone know that my Python Metadata Importer plug-in
>> was released the other day.
>>
>>  http://toxicsoftware.com/blog/index.php/weblog/
>> python_metadata_importer_101_released/
>>
>
> You could throw away like 95% of that code if you had just used  
> PyObjC to implement CMetadataImporter...  I guess I should just  
> write the code and prove it.

Yes I'm sure you could.

I posted why I didn't use PyObjC in a previous post.

When I started the project PyObjC wasn't yet officially released for  
10.4, also I didn't think either making PyObjC a requirement or  
bundling a portion of it with my importer was particularly useful.

> Also, I have seen that the compiler module is orders of magnitude  
> slower than the parser module and will die various deaths if you  
> give it really large sources, so I would recommend moving away from  
> it (even though it's far easier to use).

Indexing all the Python files on my system doesn't seem any slower  
with the current code over the previous code which just used the  
parser module (and only used it to find function & class names). It's  
hard to measure accurately because all the indexing is occurring in  
the background spotlight daemons - and I don't know what else  
Spotlight is doing at that time. I'm not overly worried about  
performance at this point - and so far it hasn't had trouble with any  
Python file I've thrown at it (including among other things PyObjC).

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

Yeah I screwed up with the code that scans functions.

There's a new version online now that fixes that.

 Jon.

On May 26, 2005, at 09:37, Nicholas Riley wrote:



> On Thu, May 26, 2005 at 12:13:02AM -0400, Jonathan Wight wrote:
>
>
>
>> Just to let everyone know that my Python Metadata Importer plug-in
>> was released the other day.
>>
>>  http://toxicsoftware.com/blog/index.php/weblog/
>> python_metadata_importer_101_released/
>>
>>
>>
>
> Since I installed this I got a continuous stream of:
>
> May 26 08:32:27 byron mdimportserver[29832]: Exception occured: ***
> -[NSCFArray addObject:]: attempt to insert nil
>
> in console.log.  I assume this will go away once the import finishes,
> but it's rather annoying at the moment.
>
> -- Nicholas Riley <[EMAIL PROTECTED]> | <http://www.uiuc.edu/ph/www/ 
> njriley>
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig
>
>
>



___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

After adding Gideon's suggestion for CRLF ending file. My importer is  
failing on only 8 files out of 3084 files.

All the failures are with Python files that try to generate the  
__version__ attribute with code instead, e.g.:

__version__ = string.split('$Revision: 1.8 $')[1]
__version__ = '$Revision: 1.6 $'[11:-2]

And so on.

My options are:

#1 Continue to fail and not process the script any further (easiest  
solution ;-)
#2 Just ignore the attribute in question.
#3 Convert the attribute into a string even though it might not make  
much sense.
#4 Execute the line and get the computed value of the attribute.

I don't really want to execute the line - who the heck knows what it  
could do. Unless anyone has any better ideas I'm just going to try  
and gracefully ignore the attribute.

 Jon.


On May 26, 2005, at 09:37, Nicholas Riley wrote:


> On Thu, May 26, 2005 at 12:13:02AM -0400, Jonathan Wight wrote:
>
>
>> Just to let everyone know that my Python Metadata Importer plug-in
>> was released the other day.
>>
>>  http://toxicsoftware.com/blog/index.php/weblog/
>> python_metadata_importer_101_released/
>>
>>
>
> Since I installed this I got a continuous stream of:
>
> May 26 08:32:27 byron mdimportserver[29832]: Exception occured: ***
> -[NSCFArray addObject:]: attempt to insert nil
>
> in console.log.  I assume this will go away once the import finishes,
> but it's rather annoying at the moment.
>
> -- 
> Nicholas Riley <[EMAIL PROTECTED]> | <http://www.uiuc.edu/ph/www/ 
> njriley>
> ___
> Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
> http://mail.python.org/mailman/listinfo/pythonmac-sig
>
>


___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Python Metadata Importer

[Jonathan Wight]

On May 27, 2005, at 01:39, Ronald Oussoren wrote:

> On 26-mei-2005, at 16:54, Jonathan Wight wrote:
>
>> After adding Gideon's suggestion for CRLF ending file. My importer is
>> failing on only 8 files out of 3084 files.
>>
>> All the failures are with Python files that try to generate the
>> __version__ attribute with code instead, e.g.:
>>
>> __version__ = string.split('$Revision: 1.8 $')[1]
>> __version__ = '$Revision: 1.6 $'[11:-2]
>>
>> And so on.
>>
>> My options are:
>>
>> #1 Continue to fail and not process the script any further (easiest
>> solution ;-)
>> #2 Just ignore the attribute in question.
>> #3 Convert the attribute into a string even though it might not make
>> much sense.
>> #4 Execute the line and get the computed value of the attribute.
>>
>> I don't really want to execute the line - who the heck knows what it
>> could do. Unless anyone has any better ideas I'm just going to try
>> and gracefully ignore the attribute.
>>
>
> You could try to recognize some forms of safe python code and  
> execute those.
> I'd expect that the lines you mention above would be the majority of
> computed __version__ attributes.

There's another __version__ format that seems semi-common too (well 3  
instances out of 3000 or so files ;-); a tuple of two or three integers:

__version__ = (1, 0, 0)

There's also an __date__ that someone decided to do some slicing on  
as well.

Right now - I'm just going to silently ignore any non-string  
metadata. I don't think anyone is going is to lose sleep over the  
lost metadata ;-)

Thanks for the input though.

 Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig

Re: [Pythonmac-SIG] Spotlight Importing without .py

[Jonathan Wight]

On Jan 21, 2006, at 2:01 AM, Nicholas Matsakis wrote:

> http://www.apple.com/downloads/macosx/spotlight/ 
> pythonmetadataimporter.html
>
> I really dig the Spotlight Metadata Importer for python, but it  
> only works
> for files ending in ".py" which excludes various shell scripts I have
> written that have no filename extension.  Anyone have a super elegant
> solution to this problem?
>
> Here's my thoughts on the matter.  To my knowledge, Mac OS X has only
> three ways of knowing the type of a file: the HFS type code, the  
> filename
> extension, and the presence of an executable bit on a file* (in  
> order of
> precidence). I've already nixed filename extensions and the  
> executable bit
> only seems to be able to identify something as a "Unix Executable  
> File".
> So, that leaves HFS type codes. The first question is whether this  
> is an
> HFS type code lying around from the days of yore that identified a  
> python
> executable script?  If not, would anyone mind if I registered one?  
> (Once
> we've got a suitable type code, modifying the Spotlight Metadata  
> Importer
> to claim it is no big deal).

> As a crazy alternative to all of the above, the python metadata  
> importer
> could be written to work in conjunction with spotmeta
> (http://www.fluffy.co.uk/spotmeta/) to additionally run on all text or
> executable files and only add the python metadata if the file  
> actually is
> a python file.  Not that I'm suggesting this be done or anything,  
> but I
> think its an interesting idea.  Hopefully Leopard will provide a more
> elegant solution to this problem.

Hi Nicholas,

I'm the author of Python Metadata Importer. There should already be a  
HFS type code for Python source. I can add the type code to Python  
Metadata Importer to wide its search. Getting the importer to work  
with Spotmeta is AFAICT a non-starter - all Spotmeta seems to do is  
add extra metadata to a file - it doesn't change the importer that a  
file is imported with.

Jon.
___
Pythonmac-SIG maillist  -  Pythonmac-SIG@python.org
http://mail.python.org/mailman/listinfo/pythonmac-sig