Hello Group Here is a little post How to create Warp10 + Zeppelin configuration
For proper reading, article should be available on : * https://fasar.frama.io/geekage/bigdata/warp10/Warp10Zeppelin.html Here a little How To to create a Warp10 instance with a Zeppelin server. The following configuration is for Warp10-ci docker image. It means there is a read and write token already inserted : 'readTokenCI' and 'writeTokenCI'. Warp10 and Zeppelin servers use the docker-host local mount point to store configurations and databases. We need : * a docker-compose.yml configuration file to run both server * Warp10 configuration files * Zeppelin configuration files. * the Zeppelin plugin for both Warp10 and Zeppelin ## Configuring the docker-compose The `docker-compose.yml` file is the following: ```yml version: '2' services: warp10-alldb: container_name: warp10-alldb image: warp10io/warp10:2.8.1-ci environment: - WARP10_HEAP=6g - WARP10_HEAP_MAX=6g volumes: - ./warp10-rt:/data zeppelin: # WarpScript interpreter plugin should be configured at: warp10-alldb:9377 container_name: zeppelin image: apache/zeppelin:0.8.2 environment: # Configuration are : https://zeppelin.apache.org/docs/0.8.2/setup/operation/configuration.html - ZEPPELIN_ADDR=0.0.0.0 - ZEPPELIN_PORT=8080 - ZEPPELIN_LOG_DIR=/zepp-rt/logs - ZEPPELIN_NOTEBOOK_DIR=/zepp-rt/notebook - ZEPPELIN_CONF_DIR=/zepp-rt/conf volumes: - ./zepp-rt:/zepp-rt/ - ./warp10-rt/warp10/lib:/zeppelin/interpreter/WarpScript ``` ## Get the Warp10 configuration files You can get the configuration Warp10 files with : ```bash DOCKERW_ID=$(docker run --rm -d warp10io/warp10:2.8.1-ci) echo $DOCKERW_ID # You can explore container with docker exec $DOCKERW_ID ls -l --color / # Create mandatory files mkdir -p ./warp10-rt/warp10/logs touch ./warp10-rt/warp10/logs/warp10.log touch ./warp10-rt/warp10/logs/warpscript.out mkdir -p ./warp10-rt/warp10/leveldb # Wait the time Warp10 configure himsef sleep 20 # Copy the configuration of Warp10 docker cp -L $DOCKERW_ID:/opt/warp10/etc ./warp10-rt/warp10/ docker cp -L $DOCKERW_ID:/opt/warp10/lib ./warp10-rt/warp10/ docker cp -L $DOCKERW_ID:/opt/warp10/jars ./warp10-rt/warp10/ docker cp -L $DOCKERW_ID:/opt/warp10/macros ./warp10-rt/warp10/ docker cp -L $DOCKERW_ID:/opt/warp10/warpscripts ./warp10-rt/warp10/ # Copy the configuration of Sensision mkdir warp10-rt/sensision/ docker cp -L $DOCKERW_ID:/opt/sensision/etc ./warp10-rt/sensision/ docker cp -L $DOCKERW_ID:/opt/sensision/scripts ./warp10-rt/sensision/ mkdir ./warp10-rt/sensision/{logs,metrics,queued,targets} touch ./warp10-rt/sensision/logs/sensision.pid touch ./warp10-rt/sensision/logs/sensision.log # Stop and remove the running Warp10 docker container docker stop $DOCKERW_ID ``` Edit the configuration file `./warp10-rt/warp10/etc/conf.d/80--plugins.conf` and add the following: ```properties warp10.plugin.zeppelin=io.warp10.plugins.zeppelin.ZeppelinWarp10Plugin ``` Edit the configuration file `./warp10-rt/warp10/etc/conf.d/ ` and change the following: ```properties egress.clients.expose = true ``` Source: https://warpfleet.senx.io/browse/io.warp10/warp10-plugin-zeppelin/1.0.4-uberjar ## Get the Zeppeling plugin for both Zeppeling and Warp10 ```bash wget https://repo.maven.apache.org/maven2/io/warp10/warp10-plugin-zeppelin/1.0.4-uberjar/warp10-plugin-zeppelin-1.0.4-uberjar.jar mv warp10-plugin-zeppelin-1.0.4-uberjar.jar warp10-rt/warp10/lib/ ``` ## Get the Zeppelin configuration files ```bash DOCKERZ_ID=$(docker run --rm -d -v $PWD/warp10-rt/warp10/lib:/zeppelin/interpreter/WarpScript apache/zeppelin:0.8.2) echo $DOCKERZ_ID # You can explore container with docker exec $DOCKERZ_ID ls -l --color / # Copy the configuration of Zeppelin sleep 4 mkdir -p ./zepp-rt/conf/ docker cp -L $DOCKERZ_ID:/zeppelin/conf ./zepp-rt # Stop and remove the running Warp10 docker container docker stop $DOCKERZ_ID ``` ## Docker magics ! ```bash docker-compose up ``` You will find the IP of the servers with the command line: `docker network inspect test-w10_default` You can use WarpStudio with the warp10-alldb IP on port 8081. You can use Zeppelin with the zeppelin IP on port 8080 Enjoy. ## Configure Zeppelin server Go on http://ZEPPELIN_DOCKER_IP:8080/#/interpreter and configure the interpreter WarpScript with: * Connect to existing process * Host: warp10-alldb * Port: 9377 ## In case of error You can remove all the docker jobs with: ```bash docker-compose down && docker container prune && docker network prune ``` ## Enjoy Warp10 + Zeppelin Now you can enjoy with the first script: ``` %WarpScript ZLEVELS '%09' URLDECODE 'char.tab' STORE '%0A' URLDECODE 'char.newline' STORE // Create a List of string for the result (a table) with header "ts and val" [ '#table ts' $char.tab 'value' ] '' JOIN [ SWAP ] [ [ 1000 14.2 ] $char.tab JOIN ] APPEND [ [ 2000 42.12 ] $char.tab JOIN ] APPEND [ [ 3000 33.10 ] $char.tab JOIN ] APPEND [ [ 6000 20.7 ] $char.tab JOIN ] APPEND $char.newline JOIN ``` -- You received this message because you are subscribed to the Google Groups "Warp 10 users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/warp10-users/ac55d642-c9d7-4a73-844e-75c8baa78c50n%40googlegroups.com.
