Hi,
I have nothing to go on with this new error, so I modified the client to print a stack trace on segfault. Please pull the client again, and if it fails for you again, please send me the backtrace. Best, Ivo ________________________________ From: Mihailovic Ivo <[email protected]> Sent: Thursday, March 31, 2016 11:33 AM To: kusumakar bodha Cc: Neda Nasiriani; [email protected] Subject: Re: Error media-streaming benchmark Hi, Thanks for testing the new build. Can you replicate the segfault? Does it happen for any fixed argument, for example the same number of sessions? Does it happen for all 4 clients or just one randomly? Best, Ivo ________________________________ From: kusumakar bodha <[email protected]> Sent: Thursday, March 31, 2016 10:05 AM To: Mihailovic Ivo Cc: Neda Nasiriani; [email protected] Subject: Re: Error media-streaming benchmark Hi Ivo, A new error showed up while running the benchmark: peak_hunter/launch_remote.sh: line 30: 221 Segmentation fault (core dumped) httperf --hog --server 172.18.0.2 --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result1.log --num-sessions=410 --rate=41 This error is not shown up in every run, it's coming sometimes. What could be the issue? Thank you Kusumakar On Thu, Mar 31, 2016 at 9:40 AM, kusumakar bodha <[email protected]<mailto:[email protected]>> wrote: Hi Ivo, I pulled the new client and ran it, it is working fine now. My system has 72 cores, I think that was causing the trouble. Thank you very much for the support. Kusumakar On Thu, Mar 31, 2016 at 3:38 AM, Mihailovic Ivo <[email protected]<mailto:[email protected]>> wrote: Hi Neda and Bodha, Can one of you try to pull the client again from dockerhub and run it on the machine where it was failing? It should work now. Ivo ________________________________ From: Mihailovic Ivo <[email protected]<mailto:[email protected]>> Sent: Wednesday, March 30, 2016 3:05 PM To: Neda Nasiriani; kusumakar bodha Cc: [email protected]<mailto:[email protected]> Subject: Re: Error media-streaming benchmark Actually, just remove the -DHAVE_SCHED_AFFINITY part, and keep the -DHAVE_EPOLL. Ivo ________________________________ From: Mihailovic Ivo <[email protected]<mailto:[email protected]>> Sent: Wednesday, March 30, 2016 2:41 PM To: Neda Nasiriani; kusumakar bodha Cc: [email protected]<mailto:[email protected]> Subject: Re: Error media-streaming benchmark Hi, This is the code of the getaffinity system call: SYSCALL_DEFINE3(sched_getaffinity, pid_t, pid, unsigned int, len, unsigned long __user *, user_mask_ptr) { int ret; cpumask_var_t mask; if ((len * BITS_PER_BYTE) < nr_cpu_ids) return -EINVAL; if (len & (sizeof(unsigned long)-1)) return -EINVAL; if (!alloc_cpumask_var(&mask, GFP_KERNEL)) return -ENOMEM; ret = sched_getaffinity(pid, mask); if (ret == 0) { size_t retlen = min_t(size_t, len, cpumask_size()); if (copy_to_user(user_mask_ptr, mask, retlen)) ret = -EFAULT; else ret = retlen; } free_cpumask_var(mask); return ret; } Given that videoperf passes sizeof(long) for the len parameter, this is going to fail if you run it on a machine with more than 64 cores (the first if statement in the code). Do you have more than 64 cores on your machine? Anyway, the simplest workaround for the problem is: 1) Get the sources from: https://github.com/ParsaLab/cloudsuite/tree/master/benchmarks/media-streaming 2) Comment out this line in client/files/videoperf/Makefile.in: DEFS += -DHAVE_SCHED_AFFINITY -DHAVE_EPOLL 3) Build the client yourself: docker build --rm -t cloudsuite/media-streaming:client /path/to/client Please let me know if this works for you. Best, Ivo ________________________________ From: Neda Nasiriani <[email protected]<mailto:[email protected]>> Sent: Wednesday, March 30, 2016 1:40 PM To: kusumakar bodha Cc: [email protected]<mailto:[email protected]> Subject: Re: Error media-streaming benchmark Hello kusumakar, I have the same problem and it happens on machines with more than 4 cores. If you try it on a machine with 4 or less cores this does not happen. It could be related to the cpu mask which is creating this problem for machines with >4 cores. I hope it get resolved. Please let me know if you have any findings. Thanks On Wednesday, March 30, 2016, kusumakar bodha <[email protected]<mailto:[email protected]>> wrote: Hi all, I'm trying to run the cloudsuite media-streaming benchmark and encountered the following error after I started the client. Total clients = 4 Minimum number of sessions = 25 Maximum number of sessions = 500 Launching 4 clients on localhost Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result1.log --num-sessions=25 --rate=2 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result2.log --num-sessions=25 --rate=2 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result3.log --num-sessions=25 --rate=2 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result4.log --num-sessions=25 --rate=2 sizeof(fd_set) = 128 sizeof(fd_set) = 128 sizeof(fd_set) = 128 sizeof(fd_set) = 128 httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=2 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=2 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 Run on hostname: f2ef7e2c3e74 httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=2 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=2 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 Run at: Wed Mar 30 05:02:02 2016 Run on hostname: f2ef7e2c3e74 Run at: Wed Mar 30 05:02:02 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) Run at: Wed Mar 30 05:02:02 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) Run at: Wed Mar 30 05:02:02 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) grep: /output/*: No such file or directory grep: /output/*: No such file or directory peak_hunter/launch_hunt_bin.sh: line 56: 0*100/0: division by 0 (error token is "0") Benchmark succeeded for 25 sessions Launching 4 clients on localhost Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result1.log --num-sessions=500 --rate=50 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result2.log --num-sessions=500 --rate=50 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result3.log --num-sessions=500 --rate=50 Running command httperf --hog --server streaming_server --videosesslog=[/videos/logs/cl-240p-10-10000-10-5-00.log,/videos/logs/cl-360p-10-4000-10-5-00.log,/videos/logs/cl-480p-10-1600-10-5-00.log,/videos/logs/cl-720p-10-800-10-5-00.log],[0.1,0.3,0.4,0.2],[localhost,localhost,localhost,localhost] --epoll --recv-buffer=524288 --port 80 --output-log=/output/result4.log --num-sessions=500 --rate=50 sizeof(fd_set) = 128 httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=50 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 sizeof(fd_set) = 128 Run at: Wed Mar 30 05:02:02 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=50 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 Run at: Wed Mar 30 05:02:02 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) sizeof(fd_set) = 128 httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=50 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 Run at: Wed Mar 30 05:02:03 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) sizeof(fd_set) = 128 httperf --hog --client=0/1 --server=streaming_server --port=80 --uri=/ --rate=50 --send-buffer=4096 --recv-buffer=524288 --epoll --call-stats-file= Run on hostname: f2ef7e2c3e74 Run at: Wed Mar 30 05:02:03 2016 httperf: sched_getaffinity failed, rc=-1 errno=22 (Invalid argument) grep: /output/*: No such file or directory grep: /output/*: No such file or directory peak_hunter/launch_hunt_bin.sh: line 56: 0*100/0: division by 0 (error token is "0") Benchmark succeeded for 500 sessions Maximum limit for number of sessions too low. grep: /output/*: No such file or directory (standard_in) 2: syntax error grep: /output/*: No such file or directory (standard_in) 2: syntax error grep: /output/*: No such file or directory (standard_in) 2: syntax error grep: /output/*: No such file or directory (standard_in) 2: syntax error grep: /output/*: No such file or directory (standard_in) 2: syntax error Requests: Replies: (standard_in) 1: syntax error Reply rate: (standard_in) 1: syntax error Reply time: Net I/O: What is the problem? The client is exiting immediately I'm using Docker version 1.9.0, build 76d6bc9 and kernel: 4.4.0-14-generic on ubuntu 14.04 Thank you Kusumakar -- Best Regards, Neda
