Filed bug report for confirmation: https://issues.apache.org/jira/browse/HBASE-9046
Happy to supply any additional info as requested Best, iain -- Iain Wright Cell: (562) 852-5916 <http://www.labctsi.org/> This email message is confidential, intended only for the recipient(s) named above and may contain information that is privileged, exempt from disclosure under applicable law. If you are not the intended recipient, do not disclose or disseminate the message to anyone except the intended recipient. If you have received this message in error, or are not the named recipient(s), please immediately notify the sender by return email, and delete all copies of this message. On Thu, Jul 25, 2013 at 8:25 PM, Kim Chew <kchew...@gmail.com> wrote: > Hello Ian, > > Glad I am not alone 8-) > > I ended up putting the coprocessor to a new location and all the regions > are able to run with the latest cp. > > I suspect that there are some hiccups in CoprocessorClassLoader? > > Kim > > > On Thu, Jul 25, 2013 at 7:22 PM, iain wright <iainw...@gmail.com> wrote: > > > Hi Kim, > > > > My devs ran into the same thing when doing iterative dev on coprocessors > > and were constantly re-loading them into HDFS & re loading the hbase > table. > > > > After banging my head against a keyboard for 2 days trying to find some > > kind of caching culprit (and failing), i ended up writing a little deploy > > script which appends an epoch to their jar on each deployment so there is > > no cache involved. > > > > It's hacky, but yeah, using a different co-processor name and uploading > > that to HDFS/re-enabling table should solve your problem. pasted the > script > > below for your reference. > > > > Cheers, > > > > iain wright > > sysadmin @ Telescope > > > > #! /bin/bash > > > > # must pass in a Jar > > if test -z $1 > > then > > echo "ERROR: Pass a jar as the paramter to this script" > > echo "IE: ./script.sh HbaseCoprocessors.jar" > > > > exit 1 > > fi > > > > # must be run as root > > if [ "$(id -u)" != "0" ]; then > > echo "This script must be run as root" 1>&2 > > exit 1 > > fi > > > > # increment jar > > EPOCH_NOW=`date +%s` > > mkdir ARCHIVED > > mkdir ARCHIVED/${EPOCH_NOW} > > INPUT_JAR=`echo $1 | cut -f1 -d.` > > OUTPUT_JAR=${INPUT_JAR}${EPOCH_NOW}.jar > > mv $1 ${OUTPUT_JAR} > > > > # get value of last jar loaded for hdfs cleanup > > OLD_JAR=`cat last_load.txt` > > > > # clean up > > echo "Loading ${OUTPUT_JAR} into HDFS" > > if [ "${OLD_JAR}" == "" ]; then > > echo "last_load.txt is empty, I dont know what to clean up from the > last > > run" > > else > > hadoop fs -rm /${OLD_JAR} > > fi > > > > # load into hdfs > > hadoop fs -put ${OUTPUT_JAR} /${OUTPUT_JAR} > > su -m hdfs -c 'hadoop fs -chmod 775 /${OUTPUT_JAR}' > > > > su -m hdfs -c 'hadoop fs -chown hbase:hbase /${OUTPUT_JAR}' > > hadoop fs -ls /${OUTPUT_JAR} > > echo "$OUTPUT_JAR loaded into HDFS" > > > > # load into hbase > > echo "Loading ${OUTPUT_JAR} into HBASE" > > shift > > cat > hbase_script <<- _EOF1_ > > disable 'test_table' > > disable 'test_table2' > > > > alter 'test_table', METHOD => 'table_att_unset', NAME => 'coprocessor\$1' > > alter 'test_table', METHOD => 'table_att', > > > > > 'coprocessor'=>'hdfs:///${OUTPUT_JAR}|telescope.hbase.coprocessors.test|1001|' > > > > alter 'test_table2', METHOD => 'table_att_unset', NAME => > 'coprocessor\$1' > > alter 'test_table2', METHOD => 'table_att_unset', NAME => > 'coprocessor\$2' > > alter 'test_table2', METHOD => 'table_att', > > > > > 'coprocessor'=>'hdfs:///${OUTPUT_JAR}|telescope.hbase.coprocessors.observers.Exporter1|1001|source_family=c' > > alter 'test_table2', METHOD => 'table_att', > > > > > 'coprocessor'=>'hdfs:///${OUTPUT_JAR}|telescope.hbase.coprocessors.observers.Expander2|1002|families=c' > > > > enable 'test_table' > > enable 'test_table2' > > exit > > _EOF1_ > > hbase shell hbase_script > > echo "${OUTPUT_JAR} Loaded into HBASE" > > echo "Cleaning up" > > > > # cleanup > > mv hbase_script ARCHIVED/$EPOCH_NOW/hbase_script > > mv ${OUTPUT_JAR} ARCHIVED/$EPOCH_NOW/${OUTPUT_JAR} > > echo $OUTPUT_JAR > last_load.txt > > > > > > > > -- > > Iain Wright > > Cell: (562) 852-5916 > > > > <http://www.labctsi.org/> > > This email message is confidential, intended only for the recipient(s) > > named above and may contain information that is privileged, exempt from > > disclosure under applicable law. If you are not the intended recipient, > do > > not disclose or disseminate the message to anyone except the intended > > recipient. If you have received this message in error, or are not the > named > > recipient(s), please immediately notify the sender by return email, and > > delete all copies of this message. > > > > > > On Wed, Jul 24, 2013 at 5:01 PM, Kim Chew <kchew...@gmail.com> wrote: > > > > > Sorry, I forgot to add that each region is located in a different RS. > > > > > > > > > On Wed, Jul 24, 2013 at 4:57 PM, Kim Chew <kchew...@gmail.com> wrote: > > > > > > > Hello there, > > > > > > > > I have an Endpoint CP which I deployed by changing the table schema > > using > > > > the 'alter' command in hbase shell. I stored my CP in hdfs. My table > > has > > > 4 > > > > regions, say, > > > > > > > > region-1 > > > > region-2 > > > > region-3 > > > > region-4 > > > > > > > > Before I deployed a new CP, I use this command to remove the old CP > > (Do I > > > > have to do that?) > > > > disable 'mytable' > > > > alter 'mytable', METHOD => 'table_att_unset', NAME => > > > 'coprocessor$1' > > > > enable 'mytable' > > > > > > > > > > > > I found out that while region-1 and region-2 runs the new version of > > CP, > > > > however region-3 and region-4 keeps using the older version. > Shouldn't > > > > re-enabling a table restart the regions thus reload the CPs ? Why > does > > > each > > > > region behave differently? > > > > > > > > Thanks. > > > > > > > > Kim > > > > > > > > > >
