Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
On Sat, May 19, 2012 at 1:21 AM, Bill Hart wrote: > On 18 May 2012 17:46, Dima Pasechnik wrote: >> On Friday, 18 May 2012 18:35:04 UTC+2, Bill Hart wrote: >>> On 18 May 2012 17:26, Jeroen Demeyer wrote: >>> > On 2012-05-18 17:29, Bill Hart wrote: >>> >> I'm not sure how virtual boxes work. Do they emulate the CPU as well as >>> >> the OS? >>> >> >>> >> Unfortunately, there isn't really a lot we can do if virtual box is >>> >> going out of its way to report incorrect cpu parameters through the >>> >> *assembly instruction* cpuid, if that is what is happening (I am not >>> >> sure if it is or not). >>> >> >>> >> Unfortunately, the solution does appear to be to build MPIR with >>> >> -march=native, which apparently works. >>> >> >>> >> If anyone has any deep insight into this, please let us know. Maybe it >>> >> is possible for us to do something about it upstream. I'm keen to fix >>> >> it, I just don't understand the problem well enough. >>> > >>> > I think the following /proc/cpuinfo explains what happens: >>> > >>> > processor : 0 >>> > vendor_id : GenuineIntel >>> > cpu family : 6 >>> > model : 42 >>> > model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz >>> > stepping : 7 >>> > cpu MHz : 3302.482 >>> > cache size : 6144 KB >>> > physical id : 0 >>> > siblings : 4 >>> > core id : 0 >>> > cpu cores : 4 >>> > apicid : 0 >>> > initial apicid : 0 >>> > fpu : yes >>> > fpu_exception : yes >>> > cpuid level : 5 >>> > wp : yes >>> > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >>> > mca cmov >>> > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc >>> > rep_good nopl pni ssse3 lahf_lm >>> > bogomips : 6604.96 >>> > clflush size : 64 >>> > cache_alignment : 64 >>> > address sizes : 36 bits physical, 48 bits virtual >>> > power management: >>> > >>> > According to vendor/family/model, this is a Sandy Bridge. However, the >>> > CPU features (see flags) are seriously reduced compared to a real Sandy >>> > Bridge. >>> > >>> > Compiling with -march=SOMETHING means that 3 things have to be >>> > satisfied: >>> > (1) GCC must understand the command line flag >>> > (2) The assembler must understand the instructions >>> > (3) The processor must support the instructions >>> > >>> > Based on my observations, it seems that MPIR already checks (1) and (2). >>> > With -march=native, (1) and (3) are guaranteed, but not (2). When >>> > checking CFLAGS in MPIR, would it be possible to actually *run* an >>> > executable compiled with -march=SOMETHING, thereby checking (3)? >>> > >>> >>> Configure does run actual compile tests for the various -march >>> options, I believe. But the problem (3) won't show up unless code >>> specifically designed to compile to instructions that are only >>> supported on that processor is compiled. I don't think this is >>> feasible (and it is an autotools/configure issue anyway, not MPIR). >>> >>> > Alternatively, check the feature flags from CPUID and lower the >>> > architecture based on that. >>> > >>> >>> This would be feasible I think, but basically defeats the entire >>> strategy of selecting CPUs in MPIR. >>> >>> So does virtual box emulate a CPU with reduced features or something? >> >> yes. This was discussed here and/or on sage-support... > > I don't read sage-support unfortunately. I do browse sage-devel > regularly, but sometimes miss relevant posts. (Having said that, I do > recall this issue being discussed, and had thought that it was > determined to be a bug in virtualbox.) > > Anyhow, please bear with me, as I am insanely ignorant about virtualbox. > > Can someone tell me what a usage case is for a virtual box which > claims to have a given CPU, but which actually has a different > (lesser) CPU? Would this be a standard application of the technology? > In other words, would people regularly build or use a virtualbox which > has a CPU whose features are different from the physical CPU of the > box? And would it be usual to expect some of the features of the > specified CPU to be missing. > > Is the restriction because of the native capabilities of the physical > CPU (i.e. does virtualbox always do this when the physical CPU doesn't > have the required features), or because the features of the CPU > virtualbox is claiming to provide are not emulated and may be at some > point when people write the relevant code, or because someone simply > introduced a bug into virtualbox which has it specify the incorrect > CPU (i.e. this is an isolated instance)? The restriction is not due to the physical CPU at all. The physical CPU's architecture is equal to the architecture that MPIR detects. That is not the problem. The problem is that Virtualbox does not support certain instructions inside the VM. And that means that all VMs made by people using Virtualbox will probably have this problem. The conditions for this proble
Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
On 18 May 2012 17:46, Dima Pasechnik wrote: > > > On Friday, 18 May 2012 18:35:04 UTC+2, Bill Hart wrote: >> >> On 18 May 2012 17:26, Jeroen Demeyer wrote: >> > On 2012-05-18 17:29, Bill Hart wrote: >> >> I'm not sure how virtual boxes work. Do they emulate the CPU as well as >> >> the OS? >> >> >> >> Unfortunately, there isn't really a lot we can do if virtual box is >> >> going out of its way to report incorrect cpu parameters through the >> >> *assembly instruction* cpuid, if that is what is happening (I am not >> >> sure if it is or not). >> >> >> >> Unfortunately, the solution does appear to be to build MPIR with >> >> -march=native, which apparently works. >> >> >> >> If anyone has any deep insight into this, please let us know. Maybe it >> >> is possible for us to do something about it upstream. I'm keen to fix >> >> it, I just don't understand the problem well enough. >> > >> > I think the following /proc/cpuinfo explains what happens: >> > >> > processor : 0 >> > vendor_id : GenuineIntel >> > cpu family : 6 >> > model : 42 >> > model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz >> > stepping : 7 >> > cpu MHz : 3302.482 >> > cache size : 6144 KB >> > physical id : 0 >> > siblings : 4 >> > core id : 0 >> > cpu cores : 4 >> > apicid : 0 >> > initial apicid : 0 >> > fpu : yes >> > fpu_exception : yes >> > cpuid level : 5 >> > wp : yes >> > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >> > mca cmov >> > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc >> > rep_good nopl pni ssse3 lahf_lm >> > bogomips : 6604.96 >> > clflush size : 64 >> > cache_alignment : 64 >> > address sizes : 36 bits physical, 48 bits virtual >> > power management: >> > >> > According to vendor/family/model, this is a Sandy Bridge. However, the >> > CPU features (see flags) are seriously reduced compared to a real Sandy >> > Bridge. >> > >> > Compiling with -march=SOMETHING means that 3 things have to be >> > satisfied: >> > (1) GCC must understand the command line flag >> > (2) The assembler must understand the instructions >> > (3) The processor must support the instructions >> > >> > Based on my observations, it seems that MPIR already checks (1) and (2). >> > With -march=native, (1) and (3) are guaranteed, but not (2). When >> > checking CFLAGS in MPIR, would it be possible to actually *run* an >> > executable compiled with -march=SOMETHING, thereby checking (3)? >> > >> >> Configure does run actual compile tests for the various -march >> options, I believe. But the problem (3) won't show up unless code >> specifically designed to compile to instructions that are only >> supported on that processor is compiled. I don't think this is >> feasible (and it is an autotools/configure issue anyway, not MPIR). >> >> > Alternatively, check the feature flags from CPUID and lower the >> > architecture based on that. >> > >> >> This would be feasible I think, but basically defeats the entire >> strategy of selecting CPUs in MPIR. >> >> So does virtual box emulate a CPU with reduced features or something? > > yes. This was discussed here and/or on sage-support... I don't read sage-support unfortunately. I do browse sage-devel regularly, but sometimes miss relevant posts. (Having said that, I do recall this issue being discussed, and had thought that it was determined to be a bug in virtualbox.) Anyhow, please bear with me, as I am insanely ignorant about virtualbox. Can someone tell me what a usage case is for a virtual box which claims to have a given CPU, but which actually has a different (lesser) CPU? Would this be a standard application of the technology? In other words, would people regularly build or use a virtualbox which has a CPU whose features are different from the physical CPU of the box? And would it be usual to expect some of the features of the specified CPU to be missing. Is the restriction because of the native capabilities of the physical CPU (i.e. does virtualbox always do this when the physical CPU doesn't have the required features), or because the features of the CPU virtualbox is claiming to provide are not emulated and may be at some point when people write the relevant code, or because someone simply introduced a bug into virtualbox which has it specify the incorrect CPU (i.e. this is an isolated instance)? I'm just trying to determine what sort of problem it is, so that I can figure out if this is something we ought to work around or not. > > >> >> Do they consider this a bug, do you think? > > They are pwned by Oracle now, it's unlikely they'd fix anything quickly, if > at all... I'm sorry to hear that. Bill. > > Dima > >> >> >> Bill. > > -- > You received this message because you are subscribed to the Google Groups > "mpir-devel" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/mpir-devel/-/
Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
On Friday, 18 May 2012 18:35:04 UTC+2, Bill Hart wrote: > > On 18 May 2012 17:26, Jeroen Demeyer wrote: > > On 2012-05-18 17:29, Bill Hart wrote: > >> I'm not sure how virtual boxes work. Do they emulate the CPU as well as > the OS? > >> > >> Unfortunately, there isn't really a lot we can do if virtual box is > >> going out of its way to report incorrect cpu parameters through the > >> *assembly instruction* cpuid, if that is what is happening (I am not > >> sure if it is or not). > >> > >> Unfortunately, the solution does appear to be to build MPIR with > >> -march=native, which apparently works. > >> > >> If anyone has any deep insight into this, please let us know. Maybe it > >> is possible for us to do something about it upstream. I'm keen to fix > >> it, I just don't understand the problem well enough. > > > > I think the following /proc/cpuinfo explains what happens: > > > > processor : 0 > > vendor_id : GenuineIntel > > cpu family : 6 > > model : 42 > > model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz > > stepping: 7 > > cpu MHz : 3302.482 > > cache size : 6144 KB > > physical id : 0 > > siblings: 4 > > core id : 0 > > cpu cores : 4 > > apicid : 0 > > initial apicid : 0 > > fpu : yes > > fpu_exception : yes > > cpuid level : 5 > > wp : yes > > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > mca cmov > > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc > > rep_good nopl pni ssse3 lahf_lm > > bogomips: 6604.96 > > clflush size: 64 > > cache_alignment : 64 > > address sizes : 36 bits physical, 48 bits virtual > > power management: > > > > According to vendor/family/model, this is a Sandy Bridge. However, the > > CPU features (see flags) are seriously reduced compared to a real Sandy > > Bridge. > > > > Compiling with -march=SOMETHING means that 3 things have to be > satisfied: > > (1) GCC must understand the command line flag > > (2) The assembler must understand the instructions > > (3) The processor must support the instructions > > > > Based on my observations, it seems that MPIR already checks (1) and (2). > > With -march=native, (1) and (3) are guaranteed, but not (2). When > > checking CFLAGS in MPIR, would it be possible to actually *run* an > > executable compiled with -march=SOMETHING, thereby checking (3)? > > > > Configure does run actual compile tests for the various -march > options, I believe. But the problem (3) won't show up unless code > specifically designed to compile to instructions that are only > supported on that processor is compiled. I don't think this is > feasible (and it is an autotools/configure issue anyway, not MPIR). > > > Alternatively, check the feature flags from CPUID and lower the > > architecture based on that. > > > > This would be feasible I think, but basically defeats the entire > strategy of selecting CPUs in MPIR. > > So does virtual box emulate a CPU with reduced features or something? > yes. This was discussed here and/or on sage-support... > Do they consider this a bug, do you think? > They are pwned by Oracle now, it's unlikely they'd fix anything quickly, if at all... Dima > > Bill. > -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
On 18 May 2012 17:26, Jeroen Demeyer wrote: > On 2012-05-18 17:29, Bill Hart wrote: >> I'm not sure how virtual boxes work. Do they emulate the CPU as well as the >> OS? >> >> Unfortunately, there isn't really a lot we can do if virtual box is >> going out of its way to report incorrect cpu parameters through the >> *assembly instruction* cpuid, if that is what is happening (I am not >> sure if it is or not). >> >> Unfortunately, the solution does appear to be to build MPIR with >> -march=native, which apparently works. >> >> If anyone has any deep insight into this, please let us know. Maybe it >> is possible for us to do something about it upstream. I'm keen to fix >> it, I just don't understand the problem well enough. > > I think the following /proc/cpuinfo explains what happens: > > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 42 > model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz > stepping : 7 > cpu MHz : 3302.482 > cache size : 6144 KB > physical id : 0 > siblings : 4 > core id : 0 > cpu cores : 4 > apicid : 0 > initial apicid : 0 > fpu : yes > fpu_exception : yes > cpuid level : 5 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca > cmov > pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc > rep_good nopl pni ssse3 lahf_lm > bogomips : 6604.96 > clflush size : 64 > cache_alignment : 64 > address sizes : 36 bits physical, 48 bits virtual > power management: > > According to vendor/family/model, this is a Sandy Bridge. However, the > CPU features (see flags) are seriously reduced compared to a real Sandy > Bridge. > > Compiling with -march=SOMETHING means that 3 things have to be satisfied: > (1) GCC must understand the command line flag > (2) The assembler must understand the instructions > (3) The processor must support the instructions > > Based on my observations, it seems that MPIR already checks (1) and (2). > With -march=native, (1) and (3) are guaranteed, but not (2). When > checking CFLAGS in MPIR, would it be possible to actually *run* an > executable compiled with -march=SOMETHING, thereby checking (3)? > Configure does run actual compile tests for the various -march options, I believe. But the problem (3) won't show up unless code specifically designed to compile to instructions that are only supported on that processor is compiled. I don't think this is feasible (and it is an autotools/configure issue anyway, not MPIR). > Alternatively, check the feature flags from CPUID and lower the > architecture based on that. > This would be feasible I think, but basically defeats the entire strategy of selecting CPUs in MPIR. So does virtual box emulate a CPU with reduced features or something? Do they consider this a bug, do you think? Bill. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
On 2012-05-18 17:29, Bill Hart wrote: > I'm not sure how virtual boxes work. Do they emulate the CPU as well as the > OS? > > Unfortunately, there isn't really a lot we can do if virtual box is > going out of its way to report incorrect cpu parameters through the > *assembly instruction* cpuid, if that is what is happening (I am not > sure if it is or not). > > Unfortunately, the solution does appear to be to build MPIR with > -march=native, which apparently works. > > If anyone has any deep insight into this, please let us know. Maybe it > is possible for us to do something about it upstream. I'm keen to fix > it, I just don't understand the problem well enough. I think the following /proc/cpuinfo explains what happens: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz stepping: 7 cpu MHz : 3302.482 cache size : 6144 KB physical id : 0 siblings: 4 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc rep_good nopl pni ssse3 lahf_lm bogomips: 6604.96 clflush size: 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: According to vendor/family/model, this is a Sandy Bridge. However, the CPU features (see flags) are seriously reduced compared to a real Sandy Bridge. Compiling with -march=SOMETHING means that 3 things have to be satisfied: (1) GCC must understand the command line flag (2) The assembler must understand the instructions (3) The processor must support the instructions Based on my observations, it seems that MPIR already checks (1) and (2). With -march=native, (1) and (3) are guaranteed, but not (2). When checking CFLAGS in MPIR, would it be possible to actually *run* an executable compiled with -march=SOMETHING, thereby checking (3)? Alternatively, check the feature flags from CPUID and lower the architecture based on that. Jeroen. -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
Re: [mpir-devel] Re: [sage-devel] Re: how do I build sage-5.0 on virtualbox?
I'm not sure how virtual boxes work. Do they emulate the CPU as well as the OS? Unfortunately, there isn't really a lot we can do if virtual box is going out of its way to report incorrect cpu parameters through the *assembly instruction* cpuid, if that is what is happening (I am not sure if it is or not). Unfortunately, the solution does appear to be to build MPIR with -march=native, which apparently works. If anyone has any deep insight into this, please let us know. Maybe it is possible for us to do something about it upstream. I'm keen to fix it, I just don't understand the problem well enough. Bill. On 18 May 2012 16:16, William Stein wrote: > On Fri, May 18, 2012 at 6:19 AM, Keshav Kini wrote: >> Jeroen Demeyer writes: >>> Something else to try: build with SAGE_FAT_BINARY=yes. Perhaps the >>> run-time CPU detection works better. >> >> Yup, that works. But it's still a workaround and not really a solution. >> It would be nice if I could type "make" and have everything just work. > > Building with SAGE_FAT_BINARY will result in other parts of Sage > (e.g., ATLAS) being built much less efficiently. It's far better to > use > > CFLAGS="-O2 -march=native" ./sage -i mpir > > which does seem to work. > > Many thanks for working on somehow fixing this. I think a lot of > people use virtualbox, so having MPIR building broken there is not > good... What do the MPIR dev's think is an acceptable solution here? > > -- William > >> >> -Keshav >> >> >> Join us in #sagemath on irc.freenode.net ! >> >> -- >> To post to this group, send an email to sage-devel@googlegroups.com >> To unsubscribe from this group, send an email to >> sage-devel+unsubscr...@googlegroups.com >> For more options, visit this group at >> http://groups.google.com/group/sage-devel >> URL: http://www.sagemath.org > > > > -- > William Stein > Professor of Mathematics > University of Washington > http://wstein.org > > -- > You received this message because you are subscribed to the Google Groups > "mpir-devel" group. > To post to this group, send email to mpir-de...@googlegroups.com. > To unsubscribe from this group, send email to > mpir-devel+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/mpir-devel?hl=en. > -- To post to this group, send an email to sage-devel@googlegroups.com To unsubscribe from this group, send an email to sage-devel+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-devel URL: http://www.sagemath.org
