Hi Yingchun,
Thanks for your reply.
I have try shell cli and just got the error:
***
[root@chaos incubator-pegasus]# ./run.sh shell
./pegasus_shell: error while loading shared libraries: libjvm.so: cannot open
shared object file: No such file or directory
***
I see that have a admin-cli from golang implement, So i decided to use
admin-cli instead of shell and ignore this error.
In link [0] I can not find doc on how i can start running an example, so i just
want to create a table on admin-cli first.
***
Pegasus-AdminCli-1.2.0 » create temp
error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
Pegasus-AdminCli-1.2.0 » create -p 2 -r 1 temp
error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
Pegasus-AdminCli-1.2.0 » create -p 1 -r 2 temp
error: partitions number must be a multiply of 2
***
It always say *CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})* ,
it doesn't any help for me, it would be great if the command can tell me what
should i do for it.
Luckily, have two table is exist on server.
***
Pegasus-AdminCli-1.2.0 » ls
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| ID | Name | Partitions | Unhealthy | WriteUnhealthy | ReadUnhealthy |
CreateTime | WReqRateLimit | WBytesRateLimit |
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| 1 | stat | 4 | 4 | 4 | 0 |
2022-08-29 | | |
| 2 | test | 4 | 4 | 4 | 0 |
2022-08-29 | | |
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
***
And use command of pegic run example for get set successed !
***
pegic » use test
ok
pegic » set h s world
ok
Encoding:
- HashKey: UTF8
- SortKey: UTF8
- Value: UTF8
pegic » get h s world
error: invalid usage of command 'get' (unconsumed input 'world'), try 'help'
pegic » get h s
h : s : world
TTL(Time-To-Live) is not set
Encoding:
- HashKey: UTF8
- SortKey: UTF8
- Value: UTF8
***
Thanks for work for golang command, this helped me a lot.
[0]. https://pegasus.apache.org/docs/tools/shell/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<[email protected]> |
| Date | 8/29/2022 00:25 |
| To | <[email protected]> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
1.
I have request meta server and got the response :
***
[root@chaos incubator-pegasus]# curl 192.168.56.11:34601
{
/: "Lists all supported calls",
...
/version: "Gets the server version."
}
***
Good, that is to say the cluster is running! You requested the HTTP
service, the server responded with the full requests it supports.
2,
Naturally, I begin use client to run example, i use go-client for it, and
got the error *** panic: pegasus table configuration query failed: failed
to connect table(temp): ERR_OBJECT_NOT_FOUND ***, it happen on [0].
I'm not sure if the table named 'temp' has been created or not when you use
the Go client to access it. You can use the command "ls -d" in shell tools
[1] to check it.
3.
container always download file for hadoop on the start,it's so weird for
me.
Yes, it is. The feature "Backup & Restore Pegasus data on HDFS" uses
libhdfs to operate on HDFS, in fact, we download it only for tests, we
can remove it later.
Thanks for your report!
1. https://pegasus.apache.org/docs/tools/shell/
Best regards,
Yingchun Lai
Best regards,
Yingchun Lai
On Sun, Aug 28, 2022 at 10:17 PM Lan Liang <[email protected]> wrote:
Have three container is running,But replica server and meta server have
not any log for request.
The replica container log,
******
[root@chaos incubator-pegasus]# docker ps|grep onebox
2562f51268c2 0246c7eb5547
"/entrypoint.sh repl…" 4 minutes ago Up 4 minutes
0.0.0.0:49159->34801/tcp,
:::49159->34801/tcp onebox-docker_replica1_1
21a506db99f4 zookeeper:3.4
"/docker-entrypoint.…" 4 minutes ago Up 4 minutes 2888/tcp,
3888/tcp, 0.0.0.0:49160->2181/tcp, :::49160->2181/tcp
onebox-docker_zookeeper_1
1b769b90e65f 0246c7eb5547
"/entrypoint.sh meta" 4 minutes ago Up 4 minutes
0.0.0.0:34601->34601/tcp,
:::34601->34601/tcp onebox-docker_meta1_1
[root@chaos incubator-pegasus]# docker logs -f --tail=20 256
239600K .......... .......... .......... .......... .......... 99% 588K 0s
239650K .......... .......... .......... .......... .......... 99% 243K 0s
239700K .......... .......... .......... .......... .......... 99% 9.90M 0s
239750K .......... .......... .......... .......... .......... 99% 8.90M 0s
239800K .......... .......... .......... .......... .......... 99% 15.8M 0s
239850K .......... .......... .......... .......... .......... 99% 7.62M 0s
239900K .......... .......... .......... .......... .......... 99% 19.2M 0s
239950K .......... .......... .......... .......... .......... 99% 1.86M 0s
240000K .......... .......... .......... .......... .......... 99% 1.68M 0s
240050K .......... .......... .......... .......... .......... 99% 11.0M 0s
240100K .......... .......... .......... .......... .......... 99% 434K 0s
240150K .......... .......... .......... .......... .......... 99% 10.8M 0s
240200K .......... .......... .......... .......... .......... 99% 1.89M 0s
240250K .......... .......... .......... .......... .......... 99% 627K 0s
240300K .......... .......... .......... .......... .......... 99% 2.38M 0s
240350K ..... 100%
105M=1m53s
2022-08-28 12:32:54 (2.09 MB/s) - 'hadoop-2.8.4.tar.gz' saved
[246123562/246123562]
Decompressing HDFS...
******
BTW, container always download file for hadoop on the start,it's so weird
for me.
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<[email protected]> |
| Date | 8/27/2022 19:42 |
| To | [email protected]<[email protected]> |
| Subject | Re: How can i run pegasus from container image |
Hi Yingchun,
Thanks for your confirmed.
I have run it success on the github action [0] and recompile pegasus and
got the different error.
Run command of ** docker run -v $PWD:/root/apache
apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build
-c" **,
And Got error:
The first time,
***
...
[ 76%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o]
Error 4
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
[ 80%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error
4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make: *** [all] Error 2
***
The second time,
***
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o]
Error 4
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
[ 77%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server
***
Third time
***
[ 78%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o]
Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
Error 2
make: *** [all] Error 2
***
It's working on Centos7, virtual machine of 6G4U.
Then i upgrade the machine to 10G6U and compile successed !,
Finally, ready to run pegasus, and Got nothing...
1. pack server
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh pack_server"
2. start service for pegasus
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
And return log say:
***
...
12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
12050K .......... .......... .......... .......... ....... 100%
7.11M=2.9s
2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz'
saved [12387614/12387614]
Decompressing zookeeper...
ZooKeeper JMX enabled by default
Using config:
/root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Zookeeper started at port 22181
starting server
cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server
config.ini -app_list meta &>result &
***
It working for zookeeper and have no any process about pegasus, IMO, It's
too hard to starte pegasus, this is so frustrating for me :(
[0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<[email protected]> |
| Date | 8/25/2022 23:52 |
| To | <[email protected]> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone [email protected]:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc -
skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ -
skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
output dsn.layer2.thrift
...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <[email protected]>
wrote:
* /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or
directory *.
The reason is run the source code packaged by windwos in linux, which
contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From
image of apache/pegasus:build-env-centos7.
***
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
output dsn.layer2.thrift
mv output/dsn.layer2_types.h
/root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command
not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<[email protected]> |
| Date | 8/24/2022 10:15 |
| To | [email protected]<[email protected]> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<[email protected]> |
| Date | 8/23/2022 23:43 |
| To | [email protected]<[email protected]> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of
compile-by-docker [0] to try it. And the container image
apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command
of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and
just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter:
No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
