[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364939#comment-17364939
]
acecile555 commented on MESOS-10219:
Yes sorry, just being a bit frustrated because I went through so many issue
with proper fix I think but the last one I have no idea and the code is really
complicated here so I do not think I can sort this one out without help :(
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: mesos_slave_windows_longpath.png,
> patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364913#comment-17364913
]
acecile555 edited comment on MESOS-10219 at 6/17/21, 12:46 PM:
---
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
Sadly it's still not starting and I'm running out of ideas now
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0617 12:27:04.621778 7352 exec.hpp:423] Failed to call `CreateProcess`:
mesos-executor.exe --launcher_dir=C:\mesos\bin
{noformat}
>From my understanding CreateProcessW is once again a shit Windows API function
>that doesn't allow long path. Some people suggest to use GetShortPathName to
>shorten the path, I'm trying to figure out how it works...
https://github.com/bazelbuild/bazel/issues/2181#issuecomment-270696173
One guy here is giving code snippet the shorten the path before calling
CreateProcessW:
https://stackoverflow.com/a/144932
But i'm not sure how I can integrate this code, it's failing here:
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/os/windows/exec.hpp#L349
Any help ? Mesos ? I'm getting close having a fully functionnal Mesos on
Windows, nobody is interested in, really ?
was (Author: acecile555):
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
Sadly it's still not starting and I'm running out of ideas now
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0617 12:27:04.621778 7352 exec.hpp:423] Failed to call `CreateProcess`:
mesos-executor.exe --launcher_dir=C:\mesos\bin
{noformat}
>From my understanding CreateProcessW is once again a shit Windows API function
>that doesn't allow long path. Some people suggest to use GetShortPathName to
>shorten the path, I'm trying to figure out how it works...
https://github.com/bazelbuild/bazel/issues/2181#issuecomment-270696173
Any help ? Mesos ? I'm getting close having a fully functionnal Mesos on
Windows, nobody is interested in, really ?
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: mesos_slave_windows_longpath.png,
> patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364913#comment-17364913
]
acecile555 edited comment on MESOS-10219 at 6/17/21, 12:42 PM:
---
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
Sadly it's still not starting and I'm running out of ideas now
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0617 12:27:04.621778 7352 exec.hpp:423] Failed to call `CreateProcess`:
mesos-executor.exe --launcher_dir=C:\mesos\bin
{noformat}
>From my understanding CreateProcessW is once again a shit Windows API function
>that doesn't allow long path. Some people suggest to use GetShortPathName to
>shorten the path, I'm trying to figure out how it works...
https://github.com/bazelbuild/bazel/issues/2181#issuecomment-270696173
Any help ? Mesos ? I'm getting close having a fully functionnal Mesos on
Windows, nobody is interested in, really ?
was (Author: acecile555):
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
Sadly it's still not starting and I'm running out of ideas now
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0617 12:27:04.621778 7352 exec.hpp:423] Failed to call `CreateProcess`:
mesos-executor.exe --launcher_dir=C:\mesos\bin
{noformat}
Any help ? Mesos ?
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: mesos_slave_windows_longpath.png,
> patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364913#comment-17364913
]
acecile555 edited comment on MESOS-10219 at 6/17/21, 12:30 PM:
---
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
Sadly it's still not starting and I'm running out of ideas now
{noformat}
WARNING: Logging before InitGoogleLogging() is written to STDERR
W0617 12:27:04.621778 7352 exec.hpp:423] Failed to call `CreateProcess`:
mesos-executor.exe --launcher_dir=C:\mesos\bin
{noformat}
Any help ? Mesos ?
was (Author: acecile555):
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: mesos_slave_windows_longpath.png,
> patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364913#comment-17364913
]
acecile555 edited comment on MESOS-10219 at 6/17/21, 12:29 PM:
---
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
!mesos_slave_windows_longpath.png!
was (Author: acecile555):
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: mesos_slave_windows_longpath.png,
> patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364913#comment-17364913
]
acecile555 commented on MESOS-10219:
So I ended up creating a file longPathAware.manifest:
{noformat}
http://schemas.microsoft.com/SMI/2016/WindowsSettings;>
true
{noformat}
And applied it to relevant exe file using:
{noformat}
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-agent.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-containerizer.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-executor.exe;1"
"C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64\mt.exe" -nologo
-manifest test.manifest -outputresource:"mesos-fetcher.exe;1"
{noformat}
And of course, I enabled long path support from local machine policies (you can
easily find this on Google).
It definitely helps as I can now access from HTTP interface executor
stdout/stderr which never worked before:
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364800#comment-17364800
]
acecile555 edited comment on MESOS-10219 at 6/17/21, 9:20 AM:
--
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
#include
+#include
#include
@@ -148,6 +149,7 @@ Future PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future PosixFilesystemIsolatorProcess::update(
continue;
}
-string link = path::join(info->directory, containerPath);
+string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -182,8 +184,9 @@ Future PosixFilesystemIsolatorProcess::update(
}
// Get user and group info for this task based on the task's sandbox.
- struct stat s;
- if (::stat(info->directory.c_str(), ) < 0) {
+ struct _stat64i32 s;
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::_wstat(longDirectoryPath.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Came up with this workaround, still have not much idea of what I'm doing but I
ended up with the following code by matching types according to compiler errors
:D
Still working fine with short path, does not crash anymore with long path, but
sadly, the app is still not starting:
{noformat}
Failed to chdir into current working directory
'C:\mesos\work\slaves\4e393619-8979-47af-8a29-0c6729b5b41a-S1873\frameworks\346d7333-a980-43a8-93ab-343ea12d77d7-\executors\tests_windows-azure-vm-with-a-very-very-very-very-very-very-very-very-very-very-very-very-very-long-apth.instance-135dbf9e-c872-11eb-970f-02422c91a06b._app.62\runs\7dfcc8c1-876e-46b5-b79a-9bd24b5c38ce':
The filename or extension is too long.
{noformat}
I can see the following in app stderr (so I moved one step ahead, mesos is
actually trying to start the app now).
The code crashing is here:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/launch.cpp#L1062
So if I understand correctly it should end up here and crashs (if someone can
confirm I read the code correctly and chdir is actually that call):
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/os/windows/chdir.hpp#L31
But it seems I'm not the only one complaining about SetCurrentDirectoryW
function:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/fb485167-2093-4465-a916-334882690e19/setcurrentdirectoryw-does-not-work-on-windows-7-for-long-paths?forum=vcgeneral
>From my understanding I should monify a manifest to enable longpathaware:
https://stackoverflow.com/a/44515362
Still trying to figure what this madness is about...
was (Author: acecile555):
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
#include
+#include
#include
@@ -148,6 +149,7 @@ Future PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future PosixFilesystemIsolatorProcess::update(
continue;
}
-string link = path::join(info->directory, containerPath);
+string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -182,8 +184,9 @@ Future PosixFilesystemIsolatorProcess::update(
}
// Get user and group info for this task based on the task's sandbox.
- struct stat s;
- if (::stat(info->directory.c_str(), ) < 0) {
+ struct _stat64i32 s;
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::_wstat(longDirectoryPath.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Came up with this
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364800#comment-17364800
]
acecile555 commented on MESOS-10219:
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
#include
+#include
#include
@@ -148,6 +149,7 @@ Future PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future PosixFilesystemIsolatorProcess::update(
continue;
}
-string link = path::join(info->directory, containerPath);
+string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -182,8 +184,9 @@ Future PosixFilesystemIsolatorProcess::update(
}
// Get user and group info for this task based on the task's sandbox.
- struct stat s;
- if (::stat(info->directory.c_str(), ) < 0) {
+ struct _stat64i32 s;
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::_wstat(longDirectoryPath.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Came up with this workaround, still have not much idea of what I'm doing but I
ended up with the following code by matching types according to compiler errors
:D
Still working fine with short path, does not crash anymore with long path, but
sadly, the app is still not starting:
{noformat}
Failed to chdir into current working directory
'C:\mesos\work\slaves\4e393619-8979-47af-8a29-0c6729b5b41a-S1873\frameworks\346d7333-a980-43a8-93ab-343ea12d77d7-\executors\tests_windows-azure-vm-with-a-very-very-very-very-very-very-very-very-very-very-very-very-very-long-apth.instance-135dbf9e-c872-11eb-970f-02422c91a06b._app.62\runs\7dfcc8c1-876e-46b5-b79a-9bd24b5c38ce':
The filename or extension is too long.
{noformat}
I can see the following in app stderr (so I moved one step ahead, mesos is
actually trying to start the app now).
The code crashing is here:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/launch.cpp#L1062
So if I understand correctly it should end up here and crashs (if someone can
confirm I read the code correctly and chdir is actually that call):
https://github.com/apache/mesos/blob/master/3rdparty/stout/include/stout/os/windows/chdir.hpp#L31
But it seems I'm not the only one complaining about SetCurrentDirectoryW
function:
https://social.msdn.microsoft.com/Forums/vstudio/en-US/fb485167-2093-4465-a916-334882690e19/setcurrentdirectoryw-does-not-work-on-windows-7-for-long-paths?forum=vcgeneral
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
>
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364786#comment-17364786
]
acecile555 commented on MESOS-10219:
Found a perl module implementing stat with longpath support and in my opinion
it's not relying on default stat implemention, so I have the feeling my second
option is the good one :/
https://metacpan.org/pod/Win32::LongPath
https://github.com/rdboisvert/Win32-LongPath/blob/master/xs/LongPath.xs#L389
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
>
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364784#comment-17364784
]
acecile555 commented on MESOS-10219:
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
#include
+#include
#include
@@ -148,6 +149,7 @@ Future PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future PosixFilesystemIsolatorProcess::update(
continue;
}
-string link = path::join(info->directory, containerPath);
+string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -183,7 +185,8 @@ Future PosixFilesystemIsolatorProcess::update(
// Get user and group info for this task based on the task's sandbox.
struct stat s;
- if (::stat(info->directory.c_str(), ) < 0) {
+ const std::wstring longDirectoryPath =
::internal::windows::longpath(directoryPath);
+ if (::stat(longDirectoryPath.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
Okay here is where I'm stuck.
I can convert the directory path to a longpath just fine, but when I replace
directoryPath with longDirectoryPath parameter to stat call, the build fails:
{noformat}
C:\mesos-build\mesos\src\slave\containerizer\mesos\isolators\filesystem\posix.cpp(189,6):
error C2664: 'int stat(const char *const ,stat *const )': cannot convert
argument 1 from 'const _Elem *' to 'const char
*const ' [C:\mesos-build\mesos\build2\src\mesos.vcxproj]
with
[
_Elem=wchar_t
]
C:\mesos-build\mesos\src\slave\containerizer\mesos\isolators\filesystem\posix.cpp(189,37):
message : Types pointed to are unrelated; conversion requires
reinterpret_cast, C-style cast or function-style cast [C:
\mesos-build\mesos\build2\src\mesos.vcxproj]
C:\Program Files (x86)\Windows
Kits\10\Include\10.0.19041.0\ucrt\sys/stat.h(238,39): message : see declaration
of 'stat' [C:\mesos-build\mesos\build2\src\mesos.vcxproj]
{noformat}
So my understanding is that "stat" call, cannot handle wstring parameter, which
lead me to two options:
* I need to cast back wstring to string, if it makes sense
* stat is not the proper function to handle long path, it needs to be replaced
by something else
Any idea ? :D
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364775#comment-17364775
]
acecile555 commented on MESOS-10219:
Hello [~cf.natali]
Actually I initially had the same feeling but it's expected.
Look at this file, it's the POSIX FS isolator but it's actually using
stout-wrapped os calls to make them portable:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
The windows emulator actually inherits from the POSIX one:
https://github.com/apache/mesos/blob/master/src/slave/containerizer/mesos/isolators/filesystem/windows.cpp
And the whole mesos-slave agent is working perfectly fine on Windows, until the
main directory exceeds 250 chars (can be tested easily using Marathon and
nested groups).
You're absolutely right, the error is raised by Windows agent on Windows, only
if the path is long, otherwise it's working fine, so it's definitely going
through this code.
If you look into the POSIX isolator you'll see stat is the only os call which
is not going through stout to have a windows version, and sadly I did not find
anything in stout so I think it's not implemented.
So far I've modified the code this way:
{noformat}
--- a/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
+++ b/src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
@@ -23,6 +23,7 @@
#include
#include
#include
+#include
#include
@@ -148,6 +149,7 @@ Future PosixFilesystemIsolatorProcess::update(
// container paths for volumes. This is enforced by the master. For
// those volumes, we create symlinks in the executor directory.
Resources current = info->resources;
+ const std::string directoryPath = info->directory;
// We first remove unneeded persistent volumes.
foreach (const Resource& resource, current.persistentVolumes()) {
@@ -168,7 +170,7 @@ Future PosixFilesystemIsolatorProcess::update(
continue;
}
-string link = path::join(info->directory, containerPath);
+string link = path::join(directoryPath, containerPath);
LOG(INFO) << "Removing symlink '" << link << "' for persistent volume "
<< resource << " of container " << containerId;
@@ -183,7 +185,7 @@ Future PosixFilesystemIsolatorProcess::update(
// Get user and group info for this task based on the task's sandbox.
struct stat s;
- if (::stat(info->directory.c_str(), ) < 0) {
+ if (::stat(directoryPath.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
It's seems to be building so I assume info->directory does return a string.
My next move is to pass directoryPath to stout/internal/windows/longpath to
convert it to long path, but if I recall correctly last time I tried this I
ended up with casting issue when passing the longpath variable to ::stat
Thanks a lot for your help offer, that would help a lot
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364741#comment-17364741
]
acecile555 commented on MESOS-10219:
Sadly longpath support is already enabled in Mesos, I just have the feeling
this specific line of code hasn't been done correctly.
Usually os commands are sent using stout, which has window wrapper function
calling longpath to convert the path. Here ::stat is used directly, which is,
from my understanding a non-long-path-enabled function.
My initial though was to store info->directory in a variable, pass it to
longpath and call stat with this update path, but I have absolutely no idea how
to do that :/
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17364717#comment-17364717
]
acecile555 commented on MESOS-10219:
Hello,
No problem [~apeters]
Sadly it's not working with long path and I have no idea how to fix that, the
broken code is:
{noformat}
// Get user and group info for this task based on the task's sandbox.
struct stat s;
if (::stat(info->directory.c_str(), ) < 0) {
return Failure("Failed to get ownership for '" + info->directory +
"': " + os::strerror(errno));
}
{noformat}
>From src/slave/containerizer/mesos/isolators/filesystem/posix.cpp
It's raising the exception saying the file does not exist.
My understanding is that this ::stat call is not using long path extension but
I have no idea how to fix that.
I looked into stout/os/stat but there's no implementation of regular stat call.
I also tried to pass info->directory.c_str() to longpath function to convert it
but it failed with incorrect casting.
Sadly this is way beyond my knowledge about C++ and I must admit I've resigned
to write this message because I admitted that there is no way I can fix this
myself...
I hope someone from Mesos team could have a look and give me some help, it's
probably trivial but turns into something impossible if you don't know how C++
works
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 edited comment on MESOS-10219 at 6/14/21, 4:15 PM:
--
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
# This is required because CMake scripts don't like the chocolatey patch but
detect this one just fine
choco install git
choco install patch
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
git clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
unix2dos patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
was (Author: acecile555):
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install patch
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
git clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
unix2dos patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 edited comment on MESOS-10219 at 6/14/21, 4:12 PM:
--
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install patch
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
git clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
unix2dos patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
was (Author: acecile555):
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
patch -p1 < patch_1.10.0_windows_build.diff
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 edited comment on MESOS-10219 at 6/9/21, 7:33 AM:
-
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
was (Author: acecile555):
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git heckout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 edited comment on MESOS-10219 at 6/9/21, 7:33 AM:
-
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
patch -p1 < patch_1.10.0_windows_build.diff
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
was (Author: acecile555):
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git checkout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
>
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355198#comment-17355198
]
acecile555 commented on MESOS-10219:
For the record, minimal set of binaries required to have a running Windows
executor is:
* mesos-agent.exe
* mesos-containerizer.exe
* mesos-executor.exe
* mesos-fetcher.exe
I'm not sure what others are for, but a slave can run without them.
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
>
[jira] [Comment Edited] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 edited comment on MESOS-10219 at 6/1/21, 3:42 PM:
-
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git heckout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64"
"C:\Program Files\CMake\bin\cmake.exe" --build . --config Release
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
The flag --config Release when running cmake --build is mandatory otherwise you
will create debug build depending at runtime of visual studio
(VCRUNTIME140D.dll).
That's documented here:
http://mesos.apache.org/documentation/latest/cmake-examples/#build-time-configurations
Regards
was (Author: acecile555):
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git heckout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64" -DCMAKE_BUILD_TYPE=Release
"C:\Program Files\CMake\bin\cmake.exe" --build .
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
Sadly the generated build depends on VCRUNTIME140D.dll (note the D standing for
debug) so Visual Studio has to be installed at runtime.
It seems -DCMAKE_BUILD_TYPE=Release is no-op.
Anyone having an idea about this issue ?
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
>
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355148#comment-17355148
]
acecile555 commented on MESOS-10219:
{noformat}
# Install patch from http://gnuwin32.sourceforge.net/packages/patch.htm
choco install git
choco install visualstudio2019buildtools
choco install visualstudio2019-workload-vctools
choco install curl
choco install cmake
choco install dos2unix
mkdir C:\mesos-build
cd C:\mesos-build
"C:\Program Files\Git\bin\bash.exe"
git.exe clone https://github.com/apache/mesos.git
cd mesos
git heckout 1.10.0
curl
https://issues.apache.org/jira/secure/attachment/13024729/patch_1.10.0_windows_build.diff
-o patch_1.10.0_windows_build.diff
patch -p1 < patch_1.10.0_windows_build.diff
unix2dos 3rdparty/libarchive-3.3.2.patch
mkdir build
cd build
"C:\Program Files\CMake\bin\cmake.exe" .. -G "Visual Studio 16 2019" -T
"host=x64" -DCMAKE_BUILD_TYPE=Release
"C:\Program Files\CMake\bin\cmake.exe" --build .
{noformat}
This documentation can be used to build Mesos 1.10.0 for Windows.
Sadly the generated build depends on VCRUNTIME140D.dll (note the D standing for
debug) so Visual Studio has to be installed at runtime.
It seems -DCMAKE_BUILD_TYPE=Release is no-op.
Anyone having an idea about this issue ?
Regards
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
>
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334810#comment-17334810
]
acecile555 commented on MESOS-10219:
For the record, I have been able to run several instances off a Python API in a
Windows 10 box and everything seems to work as expected
> 1.11.0 does not build on Windows
>
>
> Key: MESOS-10219
> URL: https://issues.apache.org/jira/browse/MESOS-10219
> Project: Mesos
> Issue Type: Bug
> Components: agent, build, cmake
>Affects Versions: 1.11.0
>Reporter: acecile555
>Priority: Major
> Attachments: patch_1.10.0_windows_build.diff
>
>
> Hello,
>
> I just tried building Mesos 1.11.0 on Windows and this is not working.
>
> The first issue is libarchive compilation that can be easily workarounded by
> adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
> {noformat}
> --- a/CMakeLists.txt
> +++ b/CMakeLists.txt
> @@ -137,7 +137,7 @@
># This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
># Enable level 4 C4061: The enumerate has no associated handler in a switch
># statement.
> - SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
> + #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
># Enable level 4 C4254: A larger bit field was assigned to a smaller bit
># field.
>SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
> {noformat}
> Sadly it is failing later with issue I cannot solve myself:
> {noformat}
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> qos_controller.cpp
> resource_estimator.cpp
> slave.cpp
> state.cpp
> task_status_update_manager.cpp
> sandbox.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> composing.cpp
> isolator.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> isolator_tracker.cpp
> launch.cpp
> C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
> open include file: 'csi/state.pb.h': No such file or directory (compiling
> source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> launcher.cpp
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string &,const
> std::vector> &,const
> Option,std::allocator std::string,std::string &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
> message : while trying to match the argument list '(const char [3],
> initializer list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
> error C2668: 'os::spawn': ambiguous call to overloaded function
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
> message : could be 'Option os::spawn(const std::string &,const
> std::vector> &)'
> [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
> with
> [
> T=int
> ] (compiling source file
> C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
> C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
> message : or 'Option os::spawn(const std::string
[jira] [Commented] (MESOS-10219) 1.11.0 does not build on Windows
[
https://issues.apache.org/jira/browse/MESOS-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17334692#comment-17334692
]
acecile555 commented on MESOS-10219:
So... It seems 1.10.0 tag is a lot more windows-compliant but still requires a
lots of tweaks to get built:
h4. libarchive raising C4061: The enumerate has no associated handler in a
switch
I am not sure why I'm the only one reporting this failure but there's no hope
to get it built on my Windows 10 VM (from Azure) with latest Visual Studio
2019. Libarchive definitely have enums raising this issue and for some reason
this warning is explicitly converted into an error (/we4061).
Commenting this compiler flags from libarchive-3.3.2.patch fixes it.
h4. CMake Error: Problem with archive_read_open_file(): Unrecognized archive
format
This error has been reported as
https://issues.apache.org/jira/browse/MESOS-10140
Sadly, there's no mention of the actual fix but I found it in GIT history.
Cherry-picking dc87f8ca427876060afc4033424050de1058d769 fixes it.
h4. stout raising C2668: 'os::spawn': ambiguous call to overloaded function
This error has already been reported as
https://issues.apache.org/jira/browse/MESOS-10137
It can be partially fixed by cherry-picking
da08b0cc33d3b6b70a507348783a70ac863cb1dd but this is not enough, the same
pattern has been found in another file, see my attached patch.
h4. LNK2019: unresolved external symbol "int __cdecl os::execvp(char const
*,char * const * const)"
This error has also been reported as
https://issues.apache.org/jira/browse/MESOS-10206 and I found someone fixing it
on mailing list at
[https://lists.apache.org/x/thread.html/rb54832eac67aacb963a4cc924f9376facbbbdf3ce34958fd6a437abb@%3Cdev.mesos.apache.org%3E]
Applying manual this patch fixes the issue.
So... Finally I managed to get it build (see full diff attached, from 1.10.0
tag) and it seems to be running. I still don't know if it's actually usable but
that's a first step.
{noformat}
$ src/mesos-agent.exe
--master=zk://10.99.50.1:2181,10.99.50.2:2181,10.99.50.3:2181/mesos
--work_dir=C:\\mesos --launcher_dir= C:\\Users\\earthlab\\mesos\\build\\src
I0428 12:09:08.465437 7776 status_update_manager_process.hpp:1276] Build:
2021-04-28 10:11:47 UTC by earthlab
I0428 12:09:08.481106 7776 status_update_manager_process.hpp:1277] Version:
1.10.0
I0428 12:09:08.481106 7776 status_update_manager_process.hpp:1280] Git tag:
1.10.0
I0428 12:09:08.481106 7776 status_update_manager_process.hpp:1284] Git SHA:
1fb36dcc5a0099f147cd01bd82cd7b4f0aec2256
I0428 12:09:08.512306 7776 resolver.cpp:69] Creating default secret resolver
I0428 12:09:08.512306 7776 containerizer.cpp:317] Using isolation {
windows/cpu, filesystem/windows, windows/mem, environment_secret }
I0428 12:09:08.512306 7776 provisioner.cpp:294] Using default backend 'copy'
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@753: Client
environment:zookeeper.version=zookeeper C client 3.4.8
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@759: Client
environment:host.name=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@768: Client
environment:os.name=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@769: Client
environment:os.arch=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@770: Client
environment:os.version=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@776: Client
environment:user.name=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@787: Client
environment:user.home=
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@log_env@794: Client
environment:user.dir=C:\Users\earthlab\mesos\build
2021-04-28 12:09:08,512:4324(0x1a0c):ZOO_INFO@zookeeper_init@827: Initiating
client connection, host=10.99.50.1:2181,10.99.50.2:2181,10.99.50.3:2181
sessionTimeout=1 watcher=7FF6FB260B55 sessionId=0 sessionPasswd=
context=02425B43C9E0 flags=0
2021-04-28 12:09:08,527:4324(0x2394):ZOO_INFO@check_events@1764: initiated
connection to server [10.99.50.1:2181]
[...]
I0428 12:09:08.672559 8680 containerizer.cpp:1162] Recovering isolators
I0428 12:09:08.672559 4212 containerizer.cpp:1201] Recovering provisioner
I0428 12:09:08.672559 1992 provisioner.cpp:518] Provisioner recovery complete
I0428 12:09:08.672559 1636 slave.cpp:8205] Recovering executors
I0428 12:09:08.672559 1636 slave.cpp:8358] Finished recovery
I0428 12:09:08.684085 1992 task_status_update_manager.cpp:181] Pausing sending
task status updates
I0428 12:09:08.684085 2920 slave.cpp:1471] New master detected at
master@10.99.50.20:5050
I0428 12:09:08.684085 6304 status_update_manager_process.hpp:379] Pausing
operation status update manager
I0428 12:09:08.684085 2920 slave.cpp:1525] No credentials provided. Attempting
to register without authentication
I0428 12:09:08.684085 2920 slave.cpp:1536] Detecting new master
I0428 12:09:08.778162 1636 slave.cpp:1696] Registered with master
[jira] [Created] (MESOS-10219) 1.11.0 does not build on Windows
acecile555 created MESOS-10219:
--
Summary: 1.11.0 does not build on Windows
Key: MESOS-10219
URL: https://issues.apache.org/jira/browse/MESOS-10219
Project: Mesos
Issue Type: Bug
Components: agent, build, cmake
Affects Versions: 1.11.0
Reporter: acecile555
Hello,
I just tried building Mesos 1.11.0 on Windows and this is not working.
The first issue is libarchive compilation that can be easily workarounded by
adding the following hunk to 3rdparty/libarchive-3.3.2.patch:
{noformat}
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -137,7 +137,7 @@
# This is added into CMAKE_C_FLAGS when CMAKE_BUILD_TYPE is "Debug"
# Enable level 4 C4061: The enumerate has no associated handler in a switch
# statement.
- SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
+ #SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4061")
# Enable level 4 C4254: A larger bit field was assigned to a smaller bit
# field.
SET(CMAKE_C_FLAGS_DEBUG "${CMAKE_C_FLAGS_DEBUG} /we4254")
{noformat}
Sadly it is failing later with issue I cannot solve myself:
{noformat}
C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
open include file: 'csi/state.pb.h': No such file or directory (compiling
source file C:\Users\earthlab\mesos\src\slave\csi_server.cpp)
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
qos_controller.cpp
resource_estimator.cpp
slave.cpp
state.cpp
task_status_update_manager.cpp
sandbox.cpp
C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
open include file: 'csi/state.pb.h': No such file or directory (compiling
source file C:\Users\earthlab\mesos\src\slave\slave.cpp)
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
composing.cpp
isolator.cpp
C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
open include file: 'csi/state.pb.h': No such file or directory (compiling
source file C:\Users\earthlab\mesos\src\slave\task_status_update_manager.cpp)
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
isolator_tracker.cpp
launch.cpp
C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
open include file: 'csi/state.pb.h': No such file or directory (compiling
source file C:\Users\earthlab\mesos\src\slave\containerizer\composing.cpp)
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
launcher.cpp
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(524,34): error
C2668: 'os::spawn': ambiguous call to overloaded function
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
message : could be 'Option os::spawn(const std::string &,const
std::vector> &)'
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
with
[
T=int
] (compiling source file
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
message : or 'Option os::spawn(const std::string &,const
std::vector> &,const
Option,std::allocator>>> &)'
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
with
[
T=int
] (compiling source file
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(525,75):
message : while trying to match the argument list '(const char [3], initializer
list)' [C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47): error
C2668: 'os::spawn': ambiguous call to overloaded function
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/exec.hpp(52,20):
message : could be 'Option os::spawn(const std::string &,const
std::vector> &)'
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
with
[
T=int
] (compiling source file
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
C:\Users\earthlab\mesos\3rdparty\stout\include\stout/os/windows/exec.hpp(412,20):
message : or 'Option os::spawn(const std::string &,const
std::vector> &,const
Option,std::allocator>>> &)'
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
with
[
T=int
] (compiling source file
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp)
C:\Users\earthlab\mesos\src\slave\containerizer\mesos\launch.cpp(893,47):
message : while trying to match the argument list '(const std::string,
std::vector>)'
[C:\Users\earthlab\mesos\build\src\mesos.vcxproj]
launcher_tracker.cpp
C:\Users\earthlab\mesos\src\csi/state.hpp(22,10): fatal error C1083: Cannot
open include file: 'csi/state.pb.h': No
[jira] [Commented] (MESOS-10190) libprocess fails with "Failed to obtain the IP address for " when using CNI on some hosts
[
https://issues.apache.org/jira/browse/MESOS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203933#comment-17203933
]
acecile555 commented on MESOS-10190:
Might have something here:
/run/mesos/isolators/network/cni/fef2fe5a-52a8-428d-bd8f-710fda758c9d/mesos/eth0/network.info
{
"cniVersion": "0.2.0",
"ip4": {
"ip": "10.96.0.73/32"
},
"dns": {}
}
I'm not sure what the aim of this file is, but eth0 is definitely not correct
here, as the machine is using bonding, the correct interface is bond0...
> libprocess fails with "Failed to obtain the IP address for " when using
> CNI on some hosts
> ---
>
> Key: MESOS-10190
> URL: https://issues.apache.org/jira/browse/MESOS-10190
> Project: Mesos
> Issue Type: Bug
> Components: executor
>Affects Versions: 1.9.0
>Reporter: acecile555
>Priority: Major
>
> Hello,
>
> We deployed CNI support and 3 of our hosts (all the same) are failing to
> start container with CNI enabled. The log file is:
> {noformat}
> E0917 16:58:11.481551 16770 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for '7c4beac7-5385-4dfa-845a-beb01e13c77c'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> So I tried enforcing LIBPROCESS_IP using env variable but I saw Mesos
> overwrites it. So I rebuilt Mesos with additionnal debugging and here is the
> log:
> {noformat}
> Overwriting environment variable 'LIBPROCESS_IP' from '10.99.50.3' to
> '0.0.0.0'
> E0917 16:34:49.779429 31428 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for 'de65bbd8-b237-4884-ba87-7e13cb85078f'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> According to the code, it's expected to be set to 0.0.0.0 (MESOS-5127). So I
> tried to understand why libprocess attempts to resolve a container run uuid
> instead of the hostname, here is libprocess code:
>
> {noformat}
> // Resolve the hostname if ip is 0.0.0.0 in case we actually have
> // a valid external IP address. Note that we need only one IP
> // address, so that other processes can send and receive and
> // don't get confused as to whom they are sending to.
> if (__address__.ip.isAny()) {
> char hostname[512];
> if (gethostname(hostname, sizeof(hostname)) < 0) {
> PLOG(FATAL) << "Failed to initialize, gethostname";
> }
> // Lookup an IP address of local hostname, taking the first result.
> Try ip = net::getIP(hostname, __address__.ip.family());
> if (ip.isError()) {
> EXIT(EXIT_FAILURE)
> << "Failed to obtain the IP address for '" << hostname << "';"
> << " the DNS service may not be able to resolve it: " << ip.error();
> }
> __address__.ip = ip.get();
> }
> {noformat}
>
> Well actually this is perfectly fine, except "gethostname" returns the
> container UUID instead of an valid host IP address. How is that even possible
> ?
>
> Any help would be greatly appreciated.
> Regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MESOS-10190) libprocess fails with "Failed to obtain the IP address for " when using CNI on some hosts
[
https://issues.apache.org/jira/browse/MESOS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203930#comment-17203930
]
acecile555 commented on MESOS-10190:
So I checked the slave logs and managed to spot this:
I0929 15:31:31.490895 24317 cni.cpp:974] Bind mounted '/proc/30726/ns/net' to
'/run/mesos/isolators/network/cni/8abbbf69-80c6-4d5b-b424-8aeda80504c7/ns' for
container 8abbbf69-80c6-4d5b-b424-8aeda80504c7
I0929 15:31:31.503661 24310 cni.cpp:974] Bind mounted '/proc/30727/ns/net' to
'/run/mesos/isolators/network/cni/accdf85c-49da-4943-92e0-a208a40180da/ns' for
container accdf85c-49da-4943-92e0-a208a40180da
I0929 15:31:33.723649 24318 cni.cpp:1731] Unmounted the network namespace
handle
'/run/mesos/isolators/network/cni/8abbbf69-80c6-4d5b-b424-8aeda80504c7/ns' for
container 8abbbf69-80c6-4d5b-b424-8aeda80504c7
I0929 15:31:33.747712 24318 cni.cpp:1731] Unmounted the network namespace
handle
'/run/mesos/isolators/network/cni/accdf85c-49da-4943-92e0-a208a40180da/ns' for
container accdf85c-49da-4943-92e0-a208a40180da
1 seconds between creation and destroy is more than enough for me the capture
the actual content.
So here is an example of the generated files for container
fef2fe5a-52a8-428d-bd8f-710fda758c9d:
/run/mesos/isolators/network/cni/fef2fe5a-52a8-428d-bd8f-710fda758c9d/hostname
fef2fe5a-52a8-428d-bd8f-710fda758c9d
/run/mesos/isolators/network/cni/fef2fe5a-52a8-428d-bd8f-710fda758c9d/hosts
127.0.0.1 localhost
10.96.0.73 fef2fe5a-52a8-428d-bd8f-710fda758c9d
Both files look ok to me, but I'm not sure how it is "overlay-mounted" then,
add other id ?
> libprocess fails with "Failed to obtain the IP address for " when using
> CNI on some hosts
> ---
>
> Key: MESOS-10190
> URL: https://issues.apache.org/jira/browse/MESOS-10190
> Project: Mesos
> Issue Type: Bug
> Components: executor
>Affects Versions: 1.9.0
>Reporter: acecile555
>Priority: Major
>
> Hello,
>
> We deployed CNI support and 3 of our hosts (all the same) are failing to
> start container with CNI enabled. The log file is:
> {noformat}
> E0917 16:58:11.481551 16770 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for '7c4beac7-5385-4dfa-845a-beb01e13c77c'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> So I tried enforcing LIBPROCESS_IP using env variable but I saw Mesos
> overwrites it. So I rebuilt Mesos with additionnal debugging and here is the
> log:
> {noformat}
> Overwriting environment variable 'LIBPROCESS_IP' from '10.99.50.3' to
> '0.0.0.0'
> E0917 16:34:49.779429 31428 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for 'de65bbd8-b237-4884-ba87-7e13cb85078f'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> According to the code, it's expected to be set to 0.0.0.0 (MESOS-5127). So I
> tried to understand why libprocess attempts to resolve a container run uuid
> instead of the hostname, here is libprocess code:
>
> {noformat}
> // Resolve the hostname if ip is 0.0.0.0 in case we actually have
> // a valid external IP address. Note that we need only one IP
> // address, so that other processes can send and receive and
> // don't get confused as to whom they are sending to.
> if (__address__.ip.isAny()) {
> char hostname[512];
> if (gethostname(hostname, sizeof(hostname)) < 0) {
> PLOG(FATAL) << "Failed to initialize, gethostname";
> }
> // Lookup an IP address of local hostname, taking the first result.
> Try ip = net::getIP(hostname, __address__.ip.family());
> if (ip.isError()) {
> EXIT(EXIT_FAILURE)
> << "Failed to obtain the IP address for '" << hostname << "';"
> << " the DNS service may not be able to resolve it: " << ip.error();
> }
> __address__.ip = ip.get();
> }
> {noformat}
>
> Well actually this is perfectly fine, except "gethostname" returns the
> container UUID instead of an valid host IP address. How is that even possible
> ?
>
> Any help would be greatly appreciated.
> Regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MESOS-10190) libprocess fails with "Failed to obtain the IP address for " when using CNI on some hosts
[
https://issues.apache.org/jira/browse/MESOS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17203878#comment-17203878
]
acecile555 commented on MESOS-10190:
Hello [~qianzhang]
How can I check "mount namespace" content ? Because the container fails to
start so I guess it gets deleted.
I guess the only way is to modify the code you have linked to output some
debugging information ? Sadly I'm not skilled enough to play with gdb but I can
rebuild Mesos with added debug info printed to stderr
Regards, Adam.
> libprocess fails with "Failed to obtain the IP address for " when using
> CNI on some hosts
> ---
>
> Key: MESOS-10190
> URL: https://issues.apache.org/jira/browse/MESOS-10190
> Project: Mesos
> Issue Type: Bug
> Components: executor
>Affects Versions: 1.9.0
>Reporter: acecile555
>Priority: Major
>
> Hello,
>
> We deployed CNI support and 3 of our hosts (all the same) are failing to
> start container with CNI enabled. The log file is:
> {noformat}
> E0917 16:58:11.481551 16770 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for '7c4beac7-5385-4dfa-845a-beb01e13c77c'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> So I tried enforcing LIBPROCESS_IP using env variable but I saw Mesos
> overwrites it. So I rebuilt Mesos with additionnal debugging and here is the
> log:
> {noformat}
> Overwriting environment variable 'LIBPROCESS_IP' from '10.99.50.3' to
> '0.0.0.0'
> E0917 16:34:49.779429 31428 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for 'de65bbd8-b237-4884-ba87-7e13cb85078f'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> According to the code, it's expected to be set to 0.0.0.0 (MESOS-5127). So I
> tried to understand why libprocess attempts to resolve a container run uuid
> instead of the hostname, here is libprocess code:
>
> {noformat}
> // Resolve the hostname if ip is 0.0.0.0 in case we actually have
> // a valid external IP address. Note that we need only one IP
> // address, so that other processes can send and receive and
> // don't get confused as to whom they are sending to.
> if (__address__.ip.isAny()) {
> char hostname[512];
> if (gethostname(hostname, sizeof(hostname)) < 0) {
> PLOG(FATAL) << "Failed to initialize, gethostname";
> }
> // Lookup an IP address of local hostname, taking the first result.
> Try ip = net::getIP(hostname, __address__.ip.family());
> if (ip.isError()) {
> EXIT(EXIT_FAILURE)
> << "Failed to obtain the IP address for '" << hostname << "';"
> << " the DNS service may not be able to resolve it: " << ip.error();
> }
> __address__.ip = ip.get();
> }
> {noformat}
>
> Well actually this is perfectly fine, except "gethostname" returns the
> container UUID instead of an valid host IP address. How is that even possible
> ?
>
> Any help would be greatly appreciated.
> Regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (MESOS-10190) libprocess fails with "Failed to obtain the IP address for " when using CNI on some hosts
[
https://issues.apache.org/jira/browse/MESOS-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199903#comment-17199903
]
acecile555 commented on MESOS-10190:
Hello,
I rebuilt Mesos whith a patch logging gethostname call in libprocess and in
both a working agent and a non-working one it's getting called with an uuid
related to the container, so I guess it's a normal behavior.
I'm wiling to get deeper into the debugging but I'm missing some information
now. Can someone give me a hint about how libprocess is supposed to resolve
container uuid into an IP address ?
Best regards, Adam.
> libprocess fails with "Failed to obtain the IP address for " when using
> CNI on some hosts
> ---
>
> Key: MESOS-10190
> URL: https://issues.apache.org/jira/browse/MESOS-10190
> Project: Mesos
> Issue Type: Bug
> Components: executor
>Affects Versions: 1.9.0
>Reporter: acecile555
>Priority: Major
>
> Hello,
>
> We deployed CNI support and 3 of our hosts (all the same) are failing to
> start container with CNI enabled. The log file is:
> {noformat}
> E0917 16:58:11.481551 16770 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for '7c4beac7-5385-4dfa-845a-beb01e13c77c'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> So I tried enforcing LIBPROCESS_IP using env variable but I saw Mesos
> overwrites it. So I rebuilt Mesos with additionnal debugging and here is the
> log:
> {noformat}
> Overwriting environment variable 'LIBPROCESS_IP' from '10.99.50.3' to
> '0.0.0.0'
> E0917 16:34:49.779429 31428 process.cpp:1153] EXIT with status 1: Failed to
> obtain the IP address for 'de65bbd8-b237-4884-ba87-7e13cb85078f'; the DNS
> service may not be able to resolve it: Name or service not known{noformat}
> According to the code, it's expected to be set to 0.0.0.0 (MESOS-5127). So I
> tried to understand why libprocess attempts to resolve a container run uuid
> instead of the hostname, here is libprocess code:
>
> {noformat}
> // Resolve the hostname if ip is 0.0.0.0 in case we actually have
> // a valid external IP address. Note that we need only one IP
> // address, so that other processes can send and receive and
> // don't get confused as to whom they are sending to.
> if (__address__.ip.isAny()) {
> char hostname[512];
> if (gethostname(hostname, sizeof(hostname)) < 0) {
> PLOG(FATAL) << "Failed to initialize, gethostname";
> }
> // Lookup an IP address of local hostname, taking the first result.
> Try ip = net::getIP(hostname, __address__.ip.family());
> if (ip.isError()) {
> EXIT(EXIT_FAILURE)
> << "Failed to obtain the IP address for '" << hostname << "';"
> << " the DNS service may not be able to resolve it: " << ip.error();
> }
> __address__.ip = ip.get();
> }
> {noformat}
>
> Well actually this is perfectly fine, except "gethostname" returns the
> container UUID instead of an valid host IP address. How is that even possible
> ?
>
> Any help would be greatly appreciated.
> Regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (MESOS-10190) libprocess fails with "Failed to obtain the IP address for " when using CNI on some hosts
acecile555 created MESOS-10190:
--
Summary: libprocess fails with "Failed to obtain the IP address
for " when using CNI on some hosts
Key: MESOS-10190
URL: https://issues.apache.org/jira/browse/MESOS-10190
Project: Mesos
Issue Type: Bug
Components: executor
Affects Versions: 1.9.0
Reporter: acecile555
Hello,
We deployed CNI support and 3 of our hosts (all the same) are failing to start
container with CNI enabled. The log file is:
{noformat}
E0917 16:58:11.481551 16770 process.cpp:1153] EXIT with status 1: Failed to
obtain the IP address for '7c4beac7-5385-4dfa-845a-beb01e13c77c'; the DNS
service may not be able to resolve it: Name or service not known{noformat}
So I tried enforcing LIBPROCESS_IP using env variable but I saw Mesos
overwrites it. So I rebuilt Mesos with additionnal debugging and here is the
log:
{noformat}
Overwriting environment variable 'LIBPROCESS_IP' from '10.99.50.3' to '0.0.0.0'
E0917 16:34:49.779429 31428 process.cpp:1153] EXIT with status 1: Failed to
obtain the IP address for 'de65bbd8-b237-4884-ba87-7e13cb85078f'; the DNS
service may not be able to resolve it: Name or service not known{noformat}
According to the code, it's expected to be set to 0.0.0.0 (MESOS-5127). So I
tried to understand why libprocess attempts to resolve a container run uuid
instead of the hostname, here is libprocess code:
{noformat}
// Resolve the hostname if ip is 0.0.0.0 in case we actually have
// a valid external IP address. Note that we need only one IP
// address, so that other processes can send and receive and
// don't get confused as to whom they are sending to.
if (__address__.ip.isAny()) {
char hostname[512];if (gethostname(hostname, sizeof(hostname)) < 0) {
PLOG(FATAL) << "Failed to initialize, gethostname";
}// Lookup an IP address of local hostname, taking the first result.
Try ip = net::getIP(hostname, __address__.ip.family());if
(ip.isError()) {
EXIT(EXIT_FAILURE)
<< "Failed to obtain the IP address for '" << hostname << "';"
<< " the DNS service may not be able to resolve it: " << ip.error();
}__address__.ip = ip.get();
}
{noformat}
Well actually this is perfectly fine, except "gethostname" returns the
container UUID instead of an valid host IP address. How is that even possible ?
Any help would be greatly appreciated.
Regards, Adam.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
