[jira] Commented: (SOLR-216) Improvements to solr.py
[
https://issues.apache.org/jira/browse/SOLR-216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12525590
]
Ed Summers commented on SOLR-216:
-
Thanks guys for this really nice update. I was wrestling with the current
solr.py in svn and solr v1.2, and then remembered seeing something on the list
about an updated solr client for python.
Anyhow, I was running some unicode through add() in the latest solr.py attached
to this issue and noticed that the body isn't being encoded before sending off
to httplib.HTTPConnection.request() on line 711:
self.conn.request('POST', url, body, headers)
This can result in a stack trace like this when sending utf8 over the wire:
Traceback (most recent call last):
File ./solr_index.py, line 25, in module
solr.add_many([solr_doc])
File ../../../wdl/solr.py, line 587, in add_many
return self._update(xstr)
File ../../../wdl/solr.py, line 653, in _update
request, self.xmlheaders)
File ../../../wdl/solr.py, line 711, in _post
self.conn.request('POST', url, body, headers)
File /usr/lib/python2.5/httplib.py, line 862, in request
self._send_request(method, url, body, headers)
File /usr/lib/python2.5/httplib.py, line 888, in _send_request
self.send(body)
File /usr/lib/python2.5/httplib.py, line 707, in send
self.sock.sendall(str)
File string, line 1, in sendall
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0302' in position
85: ordinal not in range(128)
Changing line 711 to:
self.conn.request('POST', url, body.encode('utf-8'), headers)
makes the problem go away...
Improvements to solr.py
---
Key: SOLR-216
URL: https://issues.apache.org/jira/browse/SOLR-216
Project: Solr
Issue Type: Improvement
Components: clients - python
Affects Versions: 1.2
Reporter: Jason Cater
Assignee: Mike Klaas
Priority: Trivial
Attachments: solr.py, solr.py
I've taken the original solr.py code and extended it to include higher-level
functions.
* Requires python 2.3+
* Supports SSL (https://) schema
* Conforms (mostly) to PEP 8 -- the Python Style Guide
* Provides a high-level results object with implicit data type conversion
* Supports batching of update commands
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Created: (SOLR-111) new repsonse classes and connection enhancements
new repsonse classes and connection enhancements Key: SOLR-111 URL: https://issues.apache.org/jira/browse/SOLR-111 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: response_connection_changes.diff Similar to Solr::Request::* a Solr::Response::* hierarchy was created as well as a Solr::Response::Base which has a factory method for creating the appropriate response based on the request type and the raw response. Also added delete(), delete_by_query(), add(), update() and query() methods to Solr::Connection. This gets a bit closer to a DSL type of syntax which doesn't require the user to know the inner workings of solrb. I adjusted README accordingly. Solr::Connection also operates with autocommit turned *on* so commit() messages are not required when doing add(), update(), delete() calls. It can be turned off if the user doesn't want they extra http traffic. Added the ability to iterate over search results. Although need to add the ability to iterate over complete results, fetching data behind the scenes as necessary. Unit tests have been added and functional tests improved. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-111) new repsonse classes and connection enhancements
[ https://issues.apache.org/jira/browse/SOLR-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-111: Attachment: response_connection_changes.diff new repsonse classes and connection enhancements Key: SOLR-111 URL: https://issues.apache.org/jira/browse/SOLR-111 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: response_connection_changes.diff Similar to Solr::Request::* a Solr::Response::* hierarchy was created as well as a Solr::Response::Base which has a factory method for creating the appropriate response based on the request type and the raw response. Also added delete(), delete_by_query(), add(), update() and query() methods to Solr::Connection. This gets a bit closer to a DSL type of syntax which doesn't require the user to know the inner workings of solrb. I adjusted README accordingly. Solr::Connection also operates with autocommit turned *on* so commit() messages are not required when doing add(), update(), delete() calls. It can be turned off if the user doesn't want they extra http traffic. Added the ability to iterate over search results. Although need to add the ability to iterate over complete results, fetching data behind the scenes as necessary. Unit tests have been added and functional tests improved. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-111) new repsonse classes and connection enhancements
[ https://issues.apache.org/jira/browse/SOLR-111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-111: Attachment: response_connection_changes.diff new repsonse classes and connection enhancements Key: SOLR-111 URL: https://issues.apache.org/jira/browse/SOLR-111 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: response_connection_changes.diff, response_connection_changes.diff Similar to Solr::Request::* a Solr::Response::* hierarchy was created as well as a Solr::Response::Base which has a factory method for creating the appropriate response based on the request type and the raw response. Also added delete(), delete_by_query(), add(), update() and query() methods to Solr::Connection. This gets a bit closer to a DSL type of syntax which doesn't require the user to know the inner workings of solrb. I adjusted README accordingly. Solr::Connection also operates with autocommit turned *on* so commit() messages are not required when doing add(), update(), delete() calls. It can be turned off if the user doesn't want they extra http traffic. Added the ability to iterate over search results. Although need to add the ability to iterate over complete results, fetching data behind the scenes as necessary. Unit tests have been added and functional tests improved. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (SOLR-101) refactored request classes
refactored request classes -- Key: SOLR-101 URL: https://issues.apache.org/jira/browse/SOLR-101 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz.local 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Major refactoring of lib/solr/request.rb: - split out different classes into Solr::Request module - all requests inherit from Solr::Request::Base - added Solr::Document and Solr::Field classes - adjusted unit tests appropriately - functional tests not working currently -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-101) refactored request classes
[ https://issues.apache.org/jira/browse/SOLR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-101: Attachment: request_refactoring.patch refactored request classes -- Key: SOLR-101 URL: https://issues.apache.org/jira/browse/SOLR-101 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz.local 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: request_refactoring.patch Major refactoring of lib/solr/request.rb: - split out different classes into Solr::Request module - all requests inherit from Solr::Request::Base - added Solr::Document and Solr::Field classes - adjusted unit tests appropriately - functional tests not working currently -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-101) refactored request classes
[ https://issues.apache.org/jira/browse/SOLR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-101: Attachment: request_refactoring_with_ping.patch refactored request classes -- Key: SOLR-101 URL: https://issues.apache.org/jira/browse/SOLR-101 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz.local 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: request_refactoring.patch, request_refactoring_with_ping.patch Major refactoring of lib/solr/request.rb: - split out different classes into Solr::Request module - all requests inherit from Solr::Request::Base - added Solr::Document and Solr::Field classes - adjusted unit tests appropriately - functional tests not working currently -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-101) refactored request classes
[ https://issues.apache.org/jira/browse/SOLR-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-101: Attachment: request_refactoring_with_ping.patch refactored request classes -- Key: SOLR-101 URL: https://issues.apache.org/jira/browse/SOLR-101 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz.local 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers Attachments: request_refactoring.patch, request_refactoring_with_ping.patch, request_refactoring_with_ping.patch Major refactoring of lib/solr/request.rb: - split out different classes into Solr::Request module - all requests inherit from Solr::Request::Base - added Solr::Document and Solr::Field classes - adjusted unit tests appropriately - functional tests not working currently -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (SOLR-97) Rakefile now supports functional and unit tests
Rakefile now supports functional and unit tests --- Key: SOLR-97 URL: https://issues.apache.org/jira/browse/SOLR-97 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin frizz 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25 19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386 Reporter: Ed Summers This patch includes modifications to support both functional and unit tests split out into separate directories like RoR. The test server activation was converted from two independent functions into a singleton class with start() and stop() methods. The functional tests have been wrapped with a ensure clause so that the solr test server will always be shut down--even if an exception was tossed during testing. By default the solr test server will not log startup messages to STDERR, however if it's desirable to see these you can: rake SOLR_CONSOLE=true -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (SOLR-94) added solr startup/shutdown to test task
added solr startup/shutdown to test task Key: SOLR-94 URL: https://issues.apache.org/jira/browse/SOLR-94 Project: Solr Issue Type: New Feature Components: clients - ruby - flare Environment: os x Reporter: Ed Summers This patch allows the rake test task to start a solr server before running tests, and shutting it down when the tests are complete. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-94) added solr startup/shutdown to test task
[ https://issues.apache.org/jira/browse/SOLR-94?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-94: --- Attachment: solr_start_stop.patch added solr startup/shutdown to test task Key: SOLR-94 URL: https://issues.apache.org/jira/browse/SOLR-94 Project: Solr Issue Type: New Feature Components: clients - ruby - flare Environment: os x Reporter: Ed Summers Attachments: solr_start_stop.patch This patch allows the rake test task to start a solr server before running tests, and shutting it down when the tests are complete. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Created: (SOLR-95) simplification of solr_server.rb
simplification of solr_server.rb Key: SOLR-95 URL: https://issues.apache.org/jira/browse/SOLR-95 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin Kernel Version 8.8.1 Reporter: Ed Summers Priority: Minor Attachments: solr_startup_simpification.diff solr_start() was using a system call to start solr, which resulted in three separate processes that needed to be killed with a process group id. This required solr_start() to disassociate itself from the parent process with setsid, and lookup the subprocesses' parent id. using exec() instead of system() makes this complexity go away fortunately. Also the solr_server.rb was renamed to test_helper.rb as it is likely to contain other methods/classes for clearing the index, etc... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (SOLR-95) simplification of solr_server.rb
[ https://issues.apache.org/jira/browse/SOLR-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ed Summers updated SOLR-95: --- Attachment: solr_startup_simpification.diff simplification of solr_server.rb Key: SOLR-95 URL: https://issues.apache.org/jira/browse/SOLR-95 Project: Solr Issue Type: Improvement Components: clients - ruby - flare Environment: Darwin Kernel Version 8.8.1 Reporter: Ed Summers Priority: Minor Attachments: solr_startup_simpification.diff solr_start() was using a system call to start solr, which resulted in three separate processes that needed to be killed with a process group id. This required solr_start() to disassociate itself from the parent process with setsid, and lookup the subprocesses' parent id. using exec() instead of system() makes this complexity go away fortunately. Also the solr_server.rb was renamed to test_helper.rb as it is likely to contain other methods/classes for clearing the index, etc... -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
