Hi Robert, That PR did the trick for me! Have left a comment there.
Thank you for the fix! Cheers, Jill -- Jill @OpenCraft <http://opencraft.com/> On Thu, Jan 12, 2017 at 7:35 AM, Robert R <[email protected]> wrote: > I added this (untested) PR with a bit more details: > https://github.com/edx/cs_comments_service/pull/220 > > > On Wednesday, January 11, 2017 at 10:00:01 AM UTC-5, Robert R wrote: >> >> Hi Jill. >> >> I think reindex is currently broken. We are working on a PR that >> switches away from Tire (used for ES), and I discovered that reindex >> happens to be broken yesterday as part of that work. This PR is prep for >> an ES upgrade, but is compatible with the current version of ES. >> >> Here is the PR, but it is not complete yet. It is probably larger than >> what you need to fix your issue. One problem that I know you will have is >> partially fixed in this commit: https://github.com/edx >> /cs_comments_service/pull/216/commits/2be90968e96af553fa9e15 >> a6181a9d051d065a40. The issue was that Rake did not have some fixes >> that were in app.rb. >> >> I can discuss more if needed. But for now, I'm going to concentrate on >> landing this work. >> >> Robert >> >> On Monday, January 9, 2017 at 10:35:21 PM UTC-5, [email protected] >> wrote: >>> >>> Hello, >>> >>> We're setting up a new edxapp instance for a client with existing >>> courses and forum posts, and are having trouble getting ElasticSearch to >>> return search results for old posts. New posts/comments are indexed just >>> fine though. >>> >>> The new edxapp instance is running Eucalyptus.2, with data migrated from >>> Dogwood. >>> >>> Here's what I've tried: >>> >>> - Ran rake search:rebuild as recommended by >>> cs_comment_service/wiki/Search-Indexes >>> <https://github.com/edx/cs_comments_service/wiki/Search-Indexes>. >>> - Had to apply this patch >>> >>> <https://github.com/open-craft/cs_comments_service/commit/d14b0284814acb7b451d497cc9bb8c8cea65a8d0> >>> to >>> get the task to run without error (will submit that upstream). >>> - Output looked promising; there were a lot of CommentThread >>> entries read from mongodb: >>> >>> forum@c:~/cs_comments_service$ rake search:rebuild >>> /edx/app/forum/cs_comments_service/lib/tasks/flags.rake:6: warning: >>> already initialized constant ROOT >>> /edx/app/forum/cs_comments_service/lib/tasks/kpis.rake:7: warning: >>> already initialized constant ROOT >>> W, [2017-01-09T04:21:07.666963 #24546] WARN -- : Overwriting >>> existing field _id in class User. >>> I, [2017-01-09T04:21:07.675829 #24546] INFO -- : Creating new index: >>> content_20170109042107... >>> I, [2017-01-09T04:21:07.787122 #24546] INFO -- : Applying index >>> mappings for CommentThread >>> I, [2017-01-09T04:21:07.833666 #24546] INFO -- : Applying index >>> mappings for Comment >>> I, [2017-01-09T04:21:07.848420 #24546] INFO -- : ...done! >>> I, [2017-01-09T04:21:07.851193 #24546] INFO -- : preparing to >>> (re)build index >>> D, [2017-01-09T04:21:07.852430 #24546] DEBUG -- : MONGODB | Adding >>> xx.xx.xx.xx:27017 to the cluster. >>> D, [2017-01-09T04:21:07.856389 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.getnonce | STARTED | {} >>> D, [2017-01-09T04:21:07.857681 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.getnonce | SUCCEEDED | >>> 0.00074798s >>> D, [2017-01-09T04:21:07.858003 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.authenticate | STARTED | {} >>> D, [2017-01-09T04:21:07.859127 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.authenticate | SUCCEEDED | >>> 0.000842973s >>> D, [2017-01-09T04:21:07.912745 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | STARTED | >>> {"count"=>"contents", "query"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$lte"=>2017-01-09 04:21:07 UTC}}} >>> D, [2017-01-09T04:21:07.916484 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | SUCCEEDED | >>> 0.003127465s >>> D, [2017-01-09T04:21:07.918459 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | STARTED | >>> {"count"=>"contents", "query"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$lte"=>2017-01-09 04:21:07 UTC}}} >>> D, [2017-01-09T04:21:07.920659 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | SUCCEEDED | >>> 0.0019071720000000002s >>> D, [2017-01-09T04:21:07.922103 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", "filter"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$lte"=>2017-01-09 04:21:07 UTC}}, >>> "skip"=>0, "limit"=>500} >>> D, [2017-01-09T04:21:07.963584 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.041098094s >>> I, [2017-01-09T04:21:07.976211 #24546] INFO -- : >>> content_20170109042107: imported 0 of 280 (0.0% complete after 0.06 seconds) >>> D, [2017-01-09T04:21:07.978606 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.979844 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.0009901159999999999s >>> D, [2017-01-09T04:21:07.984179 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.985220 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.0007970890000000001s >>> D, [2017-01-09T04:21:07.986733 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.987790 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.000801455s >>> D, [2017-01-09T04:21:07.989637 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.990676 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.000781482s >>> D, [2017-01-09T04:21:07.994360 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.995228 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.000639859s >>> D, [2017-01-09T04:21:07.996512 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> D, [2017-01-09T04:21:07.997424 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.000684286s >>> D, [2017-01-09T04:21:07.999114 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", >>> "filter"=>{"_id"=>BSON::ObjectId('5771941e6fa57b9dee000003'), >>> "_type"=>{"$in"=>["CommentThread"]}}} >>> <snip 987 DEBUG lines similar to the last two> >>> D, [2017-01-09T04:21:10.053846 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | SUCCEEDED | >>> 0.003505191s >>> D, [2017-01-09T04:21:10.055157 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", "filter"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$lte"=>2017-01-09 04:21:07 UTC}}, >>> "skip"=>500, "limit"=>500} >>> D, [2017-01-09T04:21:10.057621 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.002178538s >>> I, [2017-01-09T04:21:10.057859 #24546] INFO -- : >>> content_20170109042107: finished importing 280 documents >>> I, [2017-01-09T04:21:10.063743 #24546] INFO -- : alias already >>> exists (will move): content_20170109042021 >>> I, [2017-01-09T04:21:10.072781 #24546] INFO -- : alias "content" now >>> points to index content_20170109042107. >>> I, [2017-01-09T04:21:10.072937 #24546] INFO -- : importing any >>> documents that changed between 2017-01-09 04:21:07 UTC and now >>> D, [2017-01-09T04:21:10.074877 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | STARTED | >>> {"count"=>"contents", "query"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$gte"=>2017-01-09 04:21:07 UTC}}} >>> D, [2017-01-09T04:21:10.077733 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | SUCCEEDED | >>> 0.002418113s >>> D, [2017-01-09T04:21:10.079052 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | STARTED | >>> {"count"=>"contents", "query"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$gte"=>2017-01-09 04:21:07 UTC}}} >>> D, [2017-01-09T04:21:10.081516 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.count | SUCCEEDED | >>> 0.0021388329999999997s >>> D, [2017-01-09T04:21:10.082713 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | STARTED | >>> {"find"=>"contents", "filter"=>{"_type"=>{"$in"=>["Comment", >>> "CommentThread"]}, "updated_at"=>{"$gte"=>2017-01-09 04:21:07 UTC}}, >>> "skip"=>0, "limit"=>500} >>> D, [2017-01-09T04:21:10.085106 #24546] DEBUG -- : MONGODB | >>> xx.xx.xx.xx:27017 | c_opencraft_hosting_forum.find | SUCCEEDED | >>> 0.002042407s >>> I, [2017-01-09T04:21:10.085363 #24546] INFO -- : >>> content_20170109042107: finished importing 0 documents >>> >>> - And sure enough, the new content_20170109042107 index now has >>> the main content alias. The other indexes must be my previous >>> attempts (though I don't remember trying that many times): >>> >>> forum@c:~/cs_comments_service$ curl http://xx.xx.xx.xx:9200/_aliases >>> { >>> "content_20170108125330":{"aliases":{}}, >>> "content_20170108125001":{"aliases":{}}, >>> "content_20170108125747":{"aliases":{}}, >>> "content_20170108125315":{"aliases":{}}, >>> "courseware_index":{"aliases":{}}, >>> "content_20170109041935":{"aliases":{}}, >>> "content_20170109042107":{"aliases":{}}, >>> "content_20170108125440":{"aliases":{}}, >>> "content_20170108125305":{"aliases":{}}, >>> "content_20170108125433":{"aliases":{}}, >>> "content_20170109045546":{"aliases":{"content":{}}}, >>> "content_20170109042021":{"aliases":{}}, >>> "content_20170108125518":{"aliases":{}} >>> } >>> >>> - But there's only two entries in in the content index, and they >>> only appeared after I added a new comment to the post: >>> >>> forum@c:~/cs_comments_service$ curl >>> http://xx.xx.xx.xx:9200/content/_search >>> >>> {"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0}, >>> "hits":{"total":2,"max_score":1.0, >>> "hits":[ >>> >>> {"_index":"content_20170109045546","_type":"comment","_id":"5873153298615406ec000000","_score":1.0, >>> "_source" : >>> {"body":"foxes","course_id":"course-v1:edX+DemoX+Demo_Course","created_at":"2017-01-09T04:44:34Z","updated_at":"2017-01-09T04:44:34Z","commentable_id":"cba3e4cd91d0466b9ac50926e495b76f","comment_thread_id":"587203839861540683000000","group_id":{},"context":"course","id":"5873153298615406ec000000","_type":"comment"}}, >>> >>> {"_index":"content_20170109045546","_type":"comment_thread","_id":"587203839861540683000000","_score":1.0, >>> "_source" : {"title":"Aeroplane videotape stomach","body":"Random words >>> for the forum search >>> test","course_id":"course-v1:edX+DemoX+Demo_Course","commentable_id":"cba3e4cd91d0466b9ac50926e495b76f","created_at":"2017-01-08T09:16:51Z","updated_at":"2017-01-09T04:44:34Z","context":"course","last_activity_at":"2017-01-09T04:44:34Z","id":"587203839861540683000000","group_id":{},"votes_point":0,"author_id":"2582","thread_id":"587203839861540683000000","_type":"comment_thread"}} >>> ] >>> }} >>> >>> >>> >>> Has anyone seen this before? Any ideas? >>> >>> -- >>> Jill >>> @OpenCraft <http://opencraft.com/> >>> >> -- > You received this message because you are subscribed to the Google Groups > "General Open edX discussion" group. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/edx-code/1011dbab-cd51-4026-922e-0a6700943fea%40googlegroups.com > <https://groups.google.com/d/msgid/edx-code/1011dbab-cd51-4026-922e-0a6700943fea%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "General Open edX discussion" group. To view this discussion on the web visit https://groups.google.com/d/msgid/edx-code/CAMZ1AtJMfktKL79D4z-JFLTdNtxmw7pViTx%2BWQB%2BKEKH29xwhA%40mail.gmail.com.
