Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Thoughts on what tools to set up the tests. Run-benchmark gives
Run-time error 112 in regex-dna.
Clint wrote umake and sh for a particular job. they seem to be OK but
will need maintaining.
The existing windows based tools seem to vary in their effect and quality.
The bash under Ubuntu is Linux, predictable and well maintained.
Windows 10 users can invoke ubuntu bash (actually I can't right this
moment; has bash-under-Ubuntu on Windows been withdrawn?).
Perhaps a bash script could (with the assistance of make and other tools
accessible from bash) create a batch script sensitive to local
variables like tmp and temp. It might be able to run the make process
under Ubuntu, and only return to Windows when the customisation was
over. Obviously Windows (driven by a batch script or power-shell etc)
would need to invoke the Unicon processes, (translate, link etc)
because we don't want to cross-compile Unicon. And because we want the
Windows timings, we need Windows to run the windows executables.
And I bet there are some areas where MS has had to adapt bash; What does
pwd say? Permissions don't translate well as you noted. Also bash
will never be hosted on Windows earlier than 10. If you want to see
benchmarks for Windows 7,8,8.1 then Ubuntu bash is not the solution.
Putting some script-writing in the cloud would solve that problem, but
the complexity and maintenance issues would be greater than umake and sh
I fear.
Oh yes, I tried run-benchmark It gave an Icon run-time error. On regex-dna
I redirected the normal output. I have kept it.
Run-time error 112
File regex-dna.icn; Line 253
invalid type to size operation
offending value: &null
Traceback:
main(list_1 = [])
getavgtimes(procedure run_regexdna_sequential,"700-thou.dat") from
line 172 in run-benchmark.icn
run_regexdna_sequential(list_5068880 = ["700-thou.dat"]) from line
129 in auxiliary.icn
{*&null} from line 253 in regex-dna.icn
just in case,
C:\Users\Jay\Documents\tmp\tests\bench>dir 700*
...
17/04/2017 22:40 7,233,416 700-thou.dat
1 File(s) 7,233,416 bytes
...
jay
On 18/04/2017 22:22, Jafar Al-Gharaibeh wrote:
> The moral of the story is that if we want to be able to run the
> benchmarks and/or the standard test suite on Windows then we have to
> come up with a way around Windows limitations/restrictions. The
> benchmark and the tests are independent of other parts of the language
> so we can easily ship them separately allowing the user to place them
> in a writable location to start with and run them there.
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Unicon-group mailing list
Unicon-group@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
On Tue, Apr 18, 2017 at 5:04 PM, Jay Hammond wrote: > Thanks Jafar, for giving me the impetus to be impatient with regex-dna . > > I think regex-dna was not hanging in the classic sense but waiting on > input as it reported as soon as I pressed in the command window. > Good catch! in my quick test I quickly gave up and left it alone. What is happening is that we failed earlier in generating some of the data files. Regex-dna is fed one of these empty files which causes it block waiting for input. I looked at the makefile and generate.icn and I can some issues there running on Windows with who is creating "generate" and what name it is given and how it is invoked. In summary: data file generation doesn't work quite right and needs to be fixed. The good news is that this is just a mysterious hang that I have to spend a week chasing down :-). That definitely progressed things Jay, very much appreciated! Cheers, Jafar -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Thanks Jafar, for giving me the impetus to be impatient with regex-dna . I think regex-dna was not hanging in the classic sense but waiting on input as it reported as soon as I pressed in the command window. here is the concluding screen-scrape regex-dna 10-thou.dat 39:51.129 0.172 13901.912x reverse-complement 150-thou.dat 0.094N/A spectral-norm 1000.484N/A thread-ring 1000 N/A 0.203 -- What have I learned so far? Run the tests in a (separate) directory owned by the current user. use Clint's umake.exe and his sh.exe . use fasta to create the mil and thou.dat files. (or get that bit of the make process fixed) running regex-dna will partly be a test of the users' reaction times, until it's fixed not to require input. I hope that's progressed things. Let me know if you want more testing done. and thanks for all the fast support. Jay -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
The moral of the story is that if we want to be able to run the benchmarks and/or the standard test suite on Windows then we have to come up with a way around Windows limitations/restrictions. The benchmark and the tests are independent of other parts of the language so we can easily ship them separately allowing the user to place them in a writable location to start with and run them there. Jay, you have made very good progress! I did run the benchmark on a new Windows machine and I got to the same point at regex-dna. I think there is a problem there. It looks like the program hangs so waiting will not help. I will look into this issue, but it is going to take me some time to do that. For now, if you want to continue just edit run-benchmark and take out regex-dna. Thank you very much for the feedback. Jafar On Tue, Apr 18, 2017 at 3:55 PM, Jay Hammond wrote: > Hi Clint, > > I'm sure your guess is not wild. > > > Yesterday I tried creating c:\Users\jay\Documents\tmp\bench and running > the tests there. > > I stumbled differently, and did not document it. > > > So I''ll try again. > > As for the sh & make issues; I don't know enough about the GNU > versions. Looking at the help to sh, it seems too much like a DOS shell > to be what you are expecting. So I will continue testing with umake and > sh from the distribution. That way someone else may be able to run the > tests too. Please don't try to reproduce my problems yet. I have had > parts of the tests run and report, so I know I am close. > > > I'll start with a clean tests directory, in my (writable) space, > > > I'll run umake in > > C:\Users\Jay\Documents\tmp\tests\bench > > twice. > > first time: > > generating input files.'generate.out' is not recognized as an > internal or external command, > operable program or batch file. > .'generate.out' is not recognized as an internal or external command, > operable program or batch file. (repeated) > > done! > > > 2nd time > > C:\Users\Jay\Documents\tmp\tests\bench>umake > generating input files.done! > > > the mil and thou .dat files are still empty. > > > # add in the populated .dat files if necessary & retry. > > I ran fasta on the 17th. I've copied the mil and thou .dat files that I > populated then. > > C:\Users\Jay\Documents\tmp\tests\bench>run-test > > Word Size Main Memory C Compiler clockOS > 64 bit 7.931 GB MinGW gcc 5 2.6 GHz MS Windows > > CPU > 4x Intel(R) Core(TM) i5-4288U CPU @ 2.60GHz > > Elapsed time h:m:s |Concurrent | > benchmark |Sequential| |Concurrent| > |Performance| > concord concord.test 0.012N/A > deal 500 0.032N/A > ipxref ipxref.test 0.016N/A > queens 9 0.031N/A > rsg rsg.test 0.063N/A > binary-trees 10 0.156 0.750 0.208x > chameneos-redux 600N/A 0.062 > fannkuch 7 0.032N/A > fasta 1000 0.015N/A > k-nucleotide 25-thou.dat 1.066N/A > mandelbrot 200 1.094 0.672 1.627x > meteor-contest 1 1.843N/A > n-body 1000 0.063N/A > pidigits 500 0.015N/A > regex-dna 10-thou.dat > > 21:50 BST. > This takes time, at least 20 mins, maybe an hour. > another post later. > > > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Unicon-group mailing list > Unicon-group@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/unicon-group > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Hi Clint, I'm sure your guess is not wild. Yesterday I tried creating c:\Users\jay\Documents\tmp\bench and running the tests there. I stumbled differently, and did not document it. So I''ll try again. As for the sh & make issues; I don't know enough about the GNU versions. Looking at the help to sh, it seems too much like a DOS shell to be what you are expecting. So I will continue testing with umake and sh from the distribution. That way someone else may be able to run the tests too. Please don't try to reproduce my problems yet. I have had parts of the tests run and report, so I know I am close. I'll start with a clean tests directory, in my (writable) space, I'll run umake in C:\Users\Jay\Documents\tmp\tests\bench twice. first time: generating input files.'generate.out' is not recognized as an internal or external command, operable program or batch file. .'generate.out' is not recognized as an internal or external command, operable program or batch file. (repeated) done! 2nd time C:\Users\Jay\Documents\tmp\tests\bench>umake generating input files.done! the mil and thou .dat files are still empty. # add in the populated .dat files if necessary & retry. I ran fasta on the 17th. I've copied the mil and thou .dat files that I populated then. C:\Users\Jay\Documents\tmp\tests\bench>run-test Word Size Main Memory C Compiler clockOS 64 bit 7.931 GB MinGW gcc 5 2.6 GHz MS Windows CPU 4x Intel(R) Core(TM) i5-4288U CPU @ 2.60GHz Elapsed time h:m:s |Concurrent | benchmark |Sequential| |Concurrent| |Performance| concord concord.test 0.012N/A deal 500 0.032N/A ipxref ipxref.test 0.016N/A queens 9 0.031N/A rsg rsg.test 0.063N/A binary-trees 10 0.156 0.750 0.208x chameneos-redux 600N/A 0.062 fannkuch 7 0.032N/A fasta 1000 0.015N/A k-nucleotide 25-thou.dat 1.066N/A mandelbrot 200 1.094 0.672 1.627x meteor-contest 1 1.843N/A n-body 1000 0.063N/A pidigits 500 0.015N/A regex-dna 10-thou.dat 21:50 BST. This takes time, at least 20 mins, maybe an hour. another post later. -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Jay,
I am gonna take a wild guess here and say: Windows does not like it very much
when we try and write data files in directories that were created as part of a
Windows application install. Besides what you've perceived as a permissions
issue, I've seen really weird behavior trying to modify such directories, like
the open/write commands succeed, but the output didn't happen, or happened in
some weird shadow directory and not in the actual directory where it was
expected. Between that, and interesting differences between your GNU\Unx make
and bash, and any deficiencies that end up being in my umake and ush, we are
not in business on benchmarks built from Windows binaries, are we. Sorry.
I think we can track this down fairly swiftly. I would start by copying (cp -r?
xcopy /S?) the tests\bench directory to someplace that you can write on, like
\users\Jay\bench or whatever. From such a writable directory, we can then start
sorting the "make" issues, if they do not magically vanish due to escaping from
"Windows shadow directory hell".
Sorting the make issues may be tougher for me since I most likely do not use
whatever you use for your GNU\Unx make and bash. Portability of said tools has
been a continual joy as we move from Windows version to Windows version, and
emulation subsystem to emulation subsystem. I have eschewed Cygwin and leave
that to others to support. I have used MSYS before and am pretty OK with it. I
have had make.exe or sh.exe that worked for years suddenly fail when a new
Windows came out, so I have written umake.icn and ush.icn and am willing to
support them, but they aren't real UNIX and that will always be problematic for
some folks.
I will download Jafar's new binaries and see if I can reproduce your problems
and/or construct a recipe for you.
Cheers,
Clint
From: Jay Hammond
Sent: Tuesday, April 18, 2017 10:32:19 AM
To: unicon-group@lists.sourceforge.net
Subject: Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
I don't know make. I need a recipe. So I am floundering around. First
the setup, then what I tried.
Fresh install of Unicon 13.0 R5080 as administrator into C:\Unicon
on a windows 10 HP laptop
OS NameMicrosoft Windows 10 Home
Version10.0.14393 Build 14393
ProcessorIntel(R) Core(TM) i5-4288U CPU @ 2.60GHz, 2601 Mhz, 2
Core(s), 4 Logical Processor(s)
System type x64-based PC, 8GB Ram,
All the commands below were run in a command window "run as
administrator" (think sudo bash ) because I had permissions problems
earlier.
"which" reports that make.exe and sh.exe are found in
C:\Program Files(x86)\GNU\Unx
the readme in C:\Unicon\tests\bench\icon says that I can prepare for
benchmarking with
make benchmark
cd C:\Unicon\tests\bench\icon
make benchmark
gives
make benchmark-icont
...
translating rsg
make[2]: *** [compile-icont] Error 1
make[2]:Leaving directory `C:/Unicon/tests/bench/icon'
make[2]: *** [benchmark-icont] Error 2
make[2]:Leaving directory `C:/Unicon/tests/bench/icon'
make: *** [benchmark] Error 2
back at prompt
it made some bat files, and some .u files.
umake benchmark
gave
umake: -1 exit by "\t benchmark-icont"
no better if I repeat it.
--
back to a clean version of C:\unicon\tests
run make on c:\Unicon\tests\bench
make
makefile:198: target `Clean' given more than once in the same rule.
unicon -s -c pool.icn
process_begin: CreateProcess((null), unicon -s -c pool.icn, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [pool.u] Error 2
dir pool*.* shows pool.icn in C:\Unicon\tests\bench .
back to a clean version of C:\unicon\tests
umake test
the last error is
{&null . clone} from line 129 in umake.icn
('test' has no referent it seems)
oH WELL.
cd C:\Unicon\tests\bench
umake
generating input files.'generate.out is not recognised as an
internal or external command,
operable program or batch file.
"'generate.out' is not recognised as an internal or external command,
operable program or batch file." appears 9 times
dir *.dat shows 9 empty .dat files with [integer] || "-" || ("mil" or
"thou") || ".dat" names e.g.25-mil.dat
ipxref.dat has today's date & time size 1.6MB 2 other dat files are
dated 2016 (so must have been precomputed)
running umake again gives no errors, but the mil and thou data files are
empty as before
running a compiled fasta.icn will create the mil and thou data files.
I did that a few days ago.
I'll reinstate them and carry on in a later session.
---
jay
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
I don't know make. I need a recipe. So I am floundering around. First
the setup, then what I tried.
Fresh install of Unicon 13.0 R5080 as administrator into C:\Unicon
on a windows 10 HP laptop
OS NameMicrosoft Windows 10 Home
Version10.0.14393 Build 14393
ProcessorIntel(R) Core(TM) i5-4288U CPU @ 2.60GHz, 2601 Mhz, 2
Core(s), 4 Logical Processor(s)
System type x64-based PC, 8GB Ram,
All the commands below were run in a command window "run as
administrator" (think sudo bash ) because I had permissions problems
earlier.
"which" reports that make.exe and sh.exe are found in
C:\Program Files(x86)\GNU\Unx
the readme in C:\Unicon\tests\bench\icon says that I can prepare for
benchmarking with
make benchmark
cd C:\Unicon\tests\bench\icon
make benchmark
gives
make benchmark-icont
...
translating rsg
make[2]: *** [compile-icont] Error 1
make[2]:Leaving directory `C:/Unicon/tests/bench/icon'
make[2]: *** [benchmark-icont] Error 2
make[2]:Leaving directory `C:/Unicon/tests/bench/icon'
make: *** [benchmark] Error 2
back at prompt
it made some bat files, and some .u files.
umake benchmark
gave
umake: -1 exit by "\t benchmark-icont"
no better if I repeat it.
--
back to a clean version of C:\unicon\tests
run make on c:\Unicon\tests\bench
make
makefile:198: target `Clean' given more than once in the same rule.
unicon -s -c pool.icn
process_begin: CreateProcess((null), unicon -s -c pool.icn, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [pool.u] Error 2
dir pool*.* shows pool.icn in C:\Unicon\tests\bench .
back to a clean version of C:\unicon\tests
umake test
the last error is
{&null . clone} from line 129 in umake.icn
('test' has no referent it seems)
oH WELL.
cd C:\Unicon\tests\bench
umake
generating input files.'generate.out is not recognised as an
internal or external command,
operable program or batch file.
"'generate.out' is not recognised as an internal or external command,
operable program or batch file." appears 9 times
dir *.dat shows 9 empty .dat files with [integer] || "-" || ("mil" or
"thou") || ".dat" names e.g.25-mil.dat
ipxref.dat has today's date & time size 1.6MB 2 other dat files are
dated 2016 (so must have been precomputed)
running umake again gives no errors, but the mil and thou data files are
empty as before
running a compiled fasta.icn will create the mil and thou data files.
I did that a few days ago.
I'll reinstate them and carry on in a later session.
---
jay
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Unicon-group mailing list
Unicon-group@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/unicon-group
Re: [Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Jay, you are welcome. As this is the first time we ship the benchmarks and the test suite with the binaries I expect to see issues here and there, especially that these were not maintained on Windows. Part of adding them to the release this time around is to get feedback so that we can fix and improve things for the next round. We might also fix and update individual tests along the way which can be easily downloaded directly from svn. Now to the real issue, what are you seeing at your end, where are you stumbling? On my end with on a new machine, the first time I ran umake under c:\unicon\tests\bench It didn't build everything. I tried again and it worked. You should see data file generated for the test at the end which might take a while. If everything goes well you should be able to do "run-benchmark" to run the "standard" test. Cheers, Jafar On Mon, Apr 17, 2017 at 5:23 PM, Jay Hammond wrote: > Thank you, Jafar, for the new Unicon Windows binary release. > Windows Unicon 13.0 - R5080 > > Has anybody tried the benchmarks in the new release? Successfully? I > keep stumbling. > jay > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Unicon-group mailing list > Unicon-group@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/unicon-group > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
[Unicon-group] Windows Unicon 13.0 - R5080 benchmark suite
Thank you, Jafar, for the new Unicon Windows binary release. Windows Unicon 13.0 - R5080 Has anybody tried the benchmarks in the new release? Successfully? I keep stumbling. jay -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Unicon-group mailing list Unicon-group@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/unicon-group
