Re: [Xen-devel] Patching error while setting up COLO
Hi,
I have a question about the network setup with COLO.
so in the colo page(
http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping)
it shows a network topology graph:
master:
br0: 192.168.0.33
eth1: 192.168.1.33
eth2: 192.168.2.33
slave:
br0: 192.168.0.88
br1: no ip address
eth1: 192.168.1.88
eth2: 192.168.2.88
Just from the master and slave configuration the page provided. I cannot
see how the two servers are going to communicate with each other if the
bridge's ip is different from both eth1 and eth2. can anybody provide a
hint how this works? Thank you!
Victor
On Tue, Mar 15, 2016 at 11:06 PM, Yu-An(Victor) Chen
wrote:
> Hi Changlong,
>
> Thanks for the reply, the script works now. Now I have a question about
> the network setup: according to the website
> <http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping> you
> sent me, colo network interfaces between two servers should be set up like
> the following? is there any other detail regarding networking I might be
> missing? Thank you!
>
> master:
> br0: 192.168.0.33
> eth1: 192.168.1.33
> eth2: 192.168.2.33
>
> slave:
> br0: 192.168.0.88
> br1: no ip address
> eth1: 192.168.1.88
> eth2: 192.168.2.88
>
>
> Victor
>
> On Mon, Mar 14, 2016 at 1:36 AM, Changlong Xie
> wrote:
>
>> On 03/09/2016 06:57 AM, Yu-An(Victor) Chen wrote:
>>
>>> Sorry for the duplicated email Congyang, I forgot to replied all:
>>>
>>> Hi Congyang,
>>>
>>> Thank you for the hint, after building xen, your script works for
>>> qemu-xen!
>>>
>>> so now I am trying to set up the secondary node with the script provided
>>> by
>>> Changlong in his first reply:
>>>
>>> ---
>>> rm -f /var/log/xen/*
>>> rm -f /var/lib/xen/userdata-d.*
>>> service xencommons start
>>> modprobe xt_SECCOLO
>>>
>>>
>>>
>>> *active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
>>> tmp_disk_size=`./qemu-colo/qemu-img info $local_img |grep 'virtual size'
>>> |awk '{print $3}'`
>>> rm -rf /mnt/ramfs/*
>>> umount /mnt/ramfs/
>>> rm -rf /mnt/ramfs/
>>> mkdir /mnt/ramfs
>>> function create_image()
>>> {
>>> /root/xie/xen/tools/qemu-xen-dir/qemu-img create -f qcow2 $1
>>> $tmp_disk_size
>>> }
>>> function prepare_temp_images()
>>> {
>>> grep -q "^none /mnt/ramfs ramfs" /proc/mounts
>>> if [[ $? -ne 0 ]]; then
>>> mount -t ramfs none /mnt/ramfs/ -o size=2G
>>> fi
>>>
>>> if [[ ! -e $active_disk ]]; then
>>> create_image $active_disk
>>> fi
>>>
>>> if [[ ! -e $hidden_disk ]]; then
>>> create_image $hidden_disk
>>> fi
>>> }
>>>
>>> ---
>>>
>>> I have question about for the codes below:
>>>
>>>
>>>
>>>
>>> *active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
>>>
>>> Do I have to create my own image and put the img in that location? if so
>>> what kind of img specifically?
>>>
>>
>> The scripts will create "/mnt/ramfs/active_disk.img" and
>> "/mnt/ramfs/hidden_disk.img" automaticly. You need create Domain U image by
>> yourself
>>
>> Thanks
>> -Xie
>>
>>
>>> because when I look into /mnt/ramfs, it is an empty directory.
>>>
>>> Thank you!
>>>
>>> On Sun, Mar 6, 2016 at 5:12 PM, Wen Congyang
>>> wrote:
>>>
>>> On 03/05/2016 09:51 AM, Yu-An(Victor) Chen wrote:
>>>>
>>>>> Hi Congyang,
>>>>>
>>>>> Thanks for your reply,
>>>>>
>>>>> even with your script, and I modify the "path_to_xen_source" to point
>>>>>
>>>> where my xen directory is. I still got this error.
>>>>
>>>>>
>>>>> ERROR: User requested feature xen
>>>>> configure was not able to find it.
>>>>> Install xen devel
>>>>>
>>>>> What do you think what I am missing? Thank you!
>>>>>
>>>>
>>&
Re: [Xen-devel] Patching error while setting up COLO
Hi Changlong,
Thanks for the reply, the script works now. Now I have a question about the
network setup: according to the website
<http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping> you
sent me, colo network interfaces between two servers should be set up like
the following? is there any other detail regarding networking I might be
missing? Thank you!
master:
br0: 192.168.0.33
eth1: 192.168.1.33
eth2: 192.168.2.33
slave:
br0: 192.168.0.88
br1: no ip address
eth1: 192.168.1.88
eth2: 192.168.2.88
Victor
On Mon, Mar 14, 2016 at 1:36 AM, Changlong Xie
wrote:
> On 03/09/2016 06:57 AM, Yu-An(Victor) Chen wrote:
>
>> Sorry for the duplicated email Congyang, I forgot to replied all:
>>
>> Hi Congyang,
>>
>> Thank you for the hint, after building xen, your script works for
>> qemu-xen!
>>
>> so now I am trying to set up the secondary node with the script provided
>> by
>> Changlong in his first reply:
>>
>> ---
>> rm -f /var/log/xen/*
>> rm -f /var/lib/xen/userdata-d.*
>> service xencommons start
>> modprobe xt_SECCOLO
>>
>>
>>
>> *active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
>> tmp_disk_size=`./qemu-colo/qemu-img info $local_img |grep 'virtual size'
>> |awk '{print $3}'`
>> rm -rf /mnt/ramfs/*
>> umount /mnt/ramfs/
>> rm -rf /mnt/ramfs/
>> mkdir /mnt/ramfs
>> function create_image()
>> {
>> /root/xie/xen/tools/qemu-xen-dir/qemu-img create -f qcow2 $1
>> $tmp_disk_size
>> }
>> function prepare_temp_images()
>> {
>> grep -q "^none /mnt/ramfs ramfs" /proc/mounts
>> if [[ $? -ne 0 ]]; then
>> mount -t ramfs none /mnt/ramfs/ -o size=2G
>> fi
>>
>> if [[ ! -e $active_disk ]]; then
>> create_image $active_disk
>> fi
>>
>> if [[ ! -e $hidden_disk ]]; then
>> create_image $hidden_disk
>> fi
>> }
>>
>> ---
>>
>> I have question about for the codes below:
>>
>>
>>
>>
>> *active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
>>
>> Do I have to create my own image and put the img in that location? if so
>> what kind of img specifically?
>>
>
> The scripts will create "/mnt/ramfs/active_disk.img" and
> "/mnt/ramfs/hidden_disk.img" automaticly. You need create Domain U image by
> yourself
>
> Thanks
> -Xie
>
>
>> because when I look into /mnt/ramfs, it is an empty directory.
>>
>> Thank you!
>>
>> On Sun, Mar 6, 2016 at 5:12 PM, Wen Congyang
>> wrote:
>>
>> On 03/05/2016 09:51 AM, Yu-An(Victor) Chen wrote:
>>>
>>>> Hi Congyang,
>>>>
>>>> Thanks for your reply,
>>>>
>>>> even with your script, and I modify the "path_to_xen_source" to point
>>>>
>>> where my xen directory is. I still got this error.
>>>
>>>>
>>>> ERROR: User requested feature xen
>>>> configure was not able to find it.
>>>> Install xen devel
>>>>
>>>> What do you think what I am missing? Thank you!
>>>>
>>>
>>> Do you build xen before?
>>>
>>> Thanks
>>> Wen Congyang
>>>
>>>
>>>> Victor
>>>>
>>>>
>>>>
>>>> On Thu, Mar 3, 2016 at 6:15 PM, Wen Congyang >>>
>>> <mailto:we...@cn.fujitsu.com>> wrote:
>>>
>>>>
>>>> On 03/04/2016 10:01 AM, Yu-An(Victor) Chen wrote:
>>>> > Hi,
>>>> >
>>>> > So I git clone
>>>>
>>>
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dxen.git&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=4j1T2HKL4uKodf62b4Tz1XtOvX81uAqCqfOcD90CRAY&s=s0fo5ej8_vZ1PmOkDCuyIroS5Zi_KpDSHI8jqodSmrg&e=
>>>
>>>> >
>>>> > but i only see branch "con-xen-v2" instead of " colo-xen-v2" so I
>>>>
>>> assume I use just use con-xen-v2.
>>>
>>>> >
>>>> > But then th
Re: [Xen-devel] Patching error while setting up COLO
Sorry for the duplicated email Congyang, I forgot to replied all:
Hi Congyang,
Thank you for the hint, after building xen, your script works for qemu-xen!
so now I am trying to set up the secondary node with the script provided by
Changlong in his first reply:
---
rm -f /var/log/xen/*
rm -f /var/lib/xen/userdata-d.*
service xencommons start
modprobe xt_SECCOLO
*active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
tmp_disk_size=`./qemu-colo/qemu-img info $local_img |grep 'virtual size'
|awk '{print $3}'`
rm -rf /mnt/ramfs/*
umount /mnt/ramfs/
rm -rf /mnt/ramfs/
mkdir /mnt/ramfs
function create_image()
{
/root/xie/xen/tools/qemu-xen-dir/qemu-img create -f qcow2 $1
$tmp_disk_size
}
function prepare_temp_images()
{
grep -q "^none /mnt/ramfs ramfs" /proc/mounts
if [[ $? -ne 0 ]]; then
mount -t ramfs none /mnt/ramfs/ -o size=2G
fi
if [[ ! -e $active_disk ]]; then
create_image $active_disk
fi
if [[ ! -e $hidden_disk ]]; then
create_image $hidden_disk
fi
}
---
I have question about for the codes below:
*active_disk=/mnt/ramfs/active_disk.imghidden_disk=/mnt/ramfs/hidden_disk.imglocal_img=/root/xie/suse-64hvm.img*
Do I have to create my own image and put the img in that location? if so
what kind of img specifically?
because when I look into /mnt/ramfs, it is an empty directory.
Thank you!
On Sun, Mar 6, 2016 at 5:12 PM, Wen Congyang wrote:
> On 03/05/2016 09:51 AM, Yu-An(Victor) Chen wrote:
> > Hi Congyang,
> >
> > Thanks for your reply,
> >
> > even with your script, and I modify the "path_to_xen_source" to point
> where my xen directory is. I still got this error.
> >
> > ERROR: User requested feature xen
> >configure was not able to find it.
> >Install xen devel
> >
> > What do you think what I am missing? Thank you!
>
> Do you build xen before?
>
> Thanks
> Wen Congyang
>
> >
> > Victor
> >
> >
> >
> > On Thu, Mar 3, 2016 at 6:15 PM, Wen Congyang <mailto:we...@cn.fujitsu.com>> wrote:
> >
> > On 03/04/2016 10:01 AM, Yu-An(Victor) Chen wrote:
> > > Hi,
> > >
> > > So I git clone
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dxen.git&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=4j1T2HKL4uKodf62b4Tz1XtOvX81uAqCqfOcD90CRAY&s=s0fo5ej8_vZ1PmOkDCuyIroS5Zi_KpDSHI8jqodSmrg&e=
> > >
> > > but i only see branch "con-xen-v2" instead of " colo-xen-v2" so I
> assume I use just use con-xen-v2.
> > >
> > > But then the following step:
> > >
> > > in both ~/qemu-colo and ~/qemu-xen
> > >
> > > ./configure --enable-xen --target-list=x86_64-softmmu
> --extra-cflags="-I$path_to_xen_source/tools/include
> -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore"
> --extra-ldflags="-L$path_to_xen_source/tools/libxc
> -L$path_to_xen_source/tools/xenstore"
> >
> >
> > This command line is out of dated. The following is my building
> scripts:
> > #! /bin/bash
> >
> > path_to_xen_source=/work/src/xen
> > #./configure --enable-xen --target-list=i386-softmmu \
> > #--extra-cflags="-I$path_to_xen_source/tools/include
> -I$path_to_xen_source/tools/libxc/include
> -I$path_to_xen_source/tools/xenstore/include" \
> > #--extra-ldflags="-L$path_to_xen_source/tools/libxc
> -L$path_to_xen_source/tools/xenstore"
> >
> > extra_cflags=""
> > extra_cflags+=" -DXC_WANT_COMPAT_EVTCHN_API=1"
> > extra_cflags+=" -DXC_WANT_COMPAT_GNTTAB_API=1"
> > extra_cflags+=" -DXC_WANT_COMPAT_MAP_FOREIGN_API=1"
> > extra_cflags+=" -I$path_to_xen_source/tools/include"
> > extra_cflags+=" -I$path_to_xen_source/tools/libs/toollog/include"
> > extra_cflags+=" -I$path_to_xen_source/tools/libs/evtchn/include"
> > extra_cflags+=" -I$path_to_xen_source/tools/libs/gnttab/include"
> > extra_cflags+="
> -I$path_to_xen_source/tools/libs/foreignmemory/include"
> > extra_cflags+=" -I$path_to_xen_source/tools/libxc/include"
> > extra_cflags+=" -I$path_to_xen_source/tools/xenstore/include"
> > ex
Re: [Xen-devel] Patching error while setting up COLO
Hi Congyang, Thanks for your reply, even with your script, and I modify the "path_to_xen_source" to point where my xen directory is. I still got this error. ERROR: User requested feature xen configure was not able to find it. Install xen devel What do you think what I am missing? Thank you! Victor On Thu, Mar 3, 2016 at 6:15 PM, Wen Congyang wrote: > On 03/04/2016 10:01 AM, Yu-An(Victor) Chen wrote: > > Hi, > > > > So I git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dxen.git&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=4j1T2HKL4uKodf62b4Tz1XtOvX81uAqCqfOcD90CRAY&s=s0fo5ej8_vZ1PmOkDCuyIroS5Zi_KpDSHI8jqodSmrg&e= > > > > but i only see branch "con-xen-v2" instead of " colo-xen-v2" so I assume > I use just use con-xen-v2. > > > > But then the following step: > > > > in both ~/qemu-colo and ~/qemu-xen > > > > ./configure --enable-xen --target-list=x86_64-softmmu > --extra-cflags="-I$path_to_xen_source/tools/include > -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" > --extra-ldflags="-L$path_to_xen_source/tools/libxc > -L$path_to_xen_source/tools/xenstore" > > > This command line is out of dated. The following is my building scripts: > #! /bin/bash > > path_to_xen_source=/work/src/xen > #./configure --enable-xen --target-list=i386-softmmu \ > #--extra-cflags="-I$path_to_xen_source/tools/include > -I$path_to_xen_source/tools/libxc/include > -I$path_to_xen_source/tools/xenstore/include" \ > #--extra-ldflags="-L$path_to_xen_source/tools/libxc > -L$path_to_xen_source/tools/xenstore" > > extra_cflags="" > extra_cflags+=" -DXC_WANT_COMPAT_EVTCHN_API=1" > extra_cflags+=" -DXC_WANT_COMPAT_GNTTAB_API=1" > extra_cflags+=" -DXC_WANT_COMPAT_MAP_FOREIGN_API=1" > extra_cflags+=" -I$path_to_xen_source/tools/include" > extra_cflags+=" -I$path_to_xen_source/tools/libs/toollog/include" > extra_cflags+=" -I$path_to_xen_source/tools/libs/evtchn/include" > extra_cflags+=" -I$path_to_xen_source/tools/libs/gnttab/include" > extra_cflags+=" -I$path_to_xen_source/tools/libs/foreignmemory/include" > extra_cflags+=" -I$path_to_xen_source/tools/libxc/include" > extra_cflags+=" -I$path_to_xen_source/tools/xenstore/include" > extra_cflags+=" -I$path_to_xen_source/tools/xenstore/compat/include" > extra_cflags+=" " > > extra_ldflags="" > extra_ldflags+=" -L$path_to_xen_source/tools/libxc" > extra_ldflags+=" -L$path_to_xen_source/tools/xenstore" > extra_ldflags+=" -L$path_to_xen_source/tools/libs/evtchn" > extra_ldflags+=" -L$path_to_xen_source/tools/libs/gnttab" > extra_ldflags+=" -L$path_to_xen_source/tools/libs/foreignmemory" > extra_ldflags+=" -Wl,-rpath-link=$path_to_xen_source/tools/libs/toollog" > extra_ldflags+=" -Wl,-rpath-link=$path_to_xen_source/tools/libs/evtchn" > extra_ldflags+=" -Wl,-rpath-link=$path_to_xen_source/tools/libs/gnttab" > extra_ldflags+=" -Wl,-rpath-link=$path_to_xen_source/tools/libs/call" > extra_ldflags+=" > -Wl,-rpath-link=$path_to_xen_source/tools/libs/foreignmemory" > extra_ldflags+=" " > > ./configure --enable-xen --target-list=i386-softmmu \ > --extra-cflags="$extra_cflags" \ > --extra-ldflags="$extra_ldflags" > > if [[ $? -ne 0 ]]; then > exit 1 > fi > > #make -j8 && make clean > make -j8 > > You can find the newest building way in tools/Makefile(xen's codes): > subdir-all-qemu-xen-dir: qemu-xen-dir-find > if test -d $(QEMU_UPSTREAM_LOC) ; then \ > source=$(QEMU_UPSTREAM_LOC); \ > else \ > source=.; \ > fi; \ > cd qemu-xen-dir; \ > if $$source/scripts/tracetool.py --check-backend --backend stderr > ; then \ > enable_trace_backend='--enable-trace-backend=stderr'; \ > else \ > enable_trace_backend='' ; \ > fi ; \ > $$source/configure --enable-xen --target-list=i386-softmmu \ > $(QEMU_XEN_ENABLE_DEBUG) \ > $$enable_trace_backend \ > --prefix=$(LIBEXEC) \ > --libdir=$(LIBEXEC_LIB) \ > --includedir=$(LIBEXEC_INC) \ > .... > > Thanks > Wen Congyang > > > > > > > I got the following error message: > > >
Re: [Xen-devel] Patching error while setting up COLO
Hi, So I git clone https://github.com/wencongyang/qemu-xen.git but i only see branch "con-xen-v2" instead of " colo-xen-v2" so I assume I use just use con-xen-v2. But then the following step: in both ~/qemu-colo and ~/qemu-xen ./configure --enable-xen --target-list=x86_64-softmmu --extra-cflags="-I$path_to_xen_source/tools/include -I$path_to_xen_source/tools/libxc -I$path_to_xen_source/tools/xenstore" --extra-ldflags="-L$path_to_xen_source/tools/libxc -L$path_to_xen_source/tools/xenstore" I got the following error message: "ERROR: User requested feature xen configure was not able to find it. Install xen devel" I found out the the error came from just simply doing this: ./configure --enable-xen I am thinking the reason is because I did this step wrong: "path_to_xen_source=~/xen" Do I just simply copy and paste the above command into the terminal and execute? Thank you! Victor Thank you! On Thu, Mar 3, 2016 at 2:46 AM, Wen Congyang wrote: > On 03/03/2016 05:39 PM, Yu-An(Victor) Chen wrote: > > Hi Changlong, > > > > Thanks for the reply, > > > > Again when I am trying to do the following: > > > > 5. build qemu-colo > > 1) cd ~/qemu-colo/; *git checkout colo-xen-v2* > > * > > * > > I got this error message *"error: pathspec 'colo-xen-v2' did not match > any file(s) known to git."* Even if I do git fetch, I still get the same > error. > > > > the qemu-colo I cloned from is provided by you > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dcolo.git&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Mgaousw-OxgAf6f9NTOk2AidO8unmTx8nKwiGLUCISU&s=Tz2SiQ2gjQexttffgWiqgwj07qsfY4TpG4Hfcpo9Lco&e= > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dxen&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Mgaousw-OxgAf6f9NTOk2AidO8unmTx8nKwiGLUCISU&s=z-KexB48-yzsD9EEJ5tC3p8tHRiRi7LoUiP-gF6kKX0&e= > , not qemu-colo > > > > > Thank you! > > > > Victor > > > > > > > > On Thu, Feb 25, 2016 at 9:07 PM, Changlong Xie < > xiecl.f...@cn.fujitsu.com <mailto:xiecl.f...@cn.fujitsu.com>> wrote: > > > > On 02/26/2016 12:55 PM, Yu-An(Victor) Chen wrote: > > > > Hi Changlong, > > > > Are you suggesting I should hold off on setting up COLO for now? > > > > > > No, just following my steps. > > > > Thanks > > -Xie > > > > > > Thanks! > > > > Victor > > > > On Thu, Feb 25, 2016 at 8:19 PM, Changlong Xie < > xiecl.f...@cn.fujitsu.com <mailto:xiecl.f...@cn.fujitsu.com>> > > wrote: > > > > On 02/26/2016 11:38 AM, Yu-An(Victor) Chen wrote: > > > > Hi Changlong, > > > > Thanks for the reply! > > > > So I am trying to follow your new instructions, but when > I am trying to do > > this: > > > >cd ~/colo-proxy/; git checkout 405527cbfa9f > > > > I got the following error: > > > > "error: pathspec '405527cbfa9f' did not match any > file(s) known to git." > > > > I assume it is just a typo? Thank you! > > > > > > Hi victor > > > > Please git clone > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Pating_colo-2Dproxy_tree_changlox&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=pCAkg_8tEQmGEZZoUlyePZjK7z-6aEmp-n6UrQRLWo4&s=Ww-EAIszC-zQuVcDc4XpigwVbMG_4t2SpTg2PV6HTjM&e= > > *Notice* that, currently we implement colo proxy as a kernel > module what > > is a temporary measure. But further more we'll intergrate it > in qemu and > > drop this one, so both qemu-colo and xen-colo will share the > same proxy. > > Please don't test this colo proxy now, there maybe some > bugs, but it's > > acceptable. > > > > Thanks > > -Xie > > > > > > Victor > > > > > > > > > > > > > > > > > > > > > > > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Patching error while setting up COLO
Hi Changlong, Thanks for the reply, Again when I am trying to do the following: 5. build qemu-colo 1) cd ~/qemu-colo/; *git checkout colo-xen-v2* I got this error message *"error: pathspec 'colo-xen-v2' did not match any file(s) known to git."* Even if I do git fetch, I still get the same error. the qemu-colo I cloned from is provided by you https://github.com/wencongyang/qemu-colo.git Thank you! Victor On Thu, Feb 25, 2016 at 9:07 PM, Changlong Xie wrote: > On 02/26/2016 12:55 PM, Yu-An(Victor) Chen wrote: > >> Hi Changlong, >> >> Are you suggesting I should hold off on setting up COLO for now? >> >> > No, just following my steps. > > Thanks > -Xie > > > Thanks! >> >> Victor >> >> On Thu, Feb 25, 2016 at 8:19 PM, Changlong Xie > > >> wrote: >> >> On 02/26/2016 11:38 AM, Yu-An(Victor) Chen wrote: >>> >>> Hi Changlong, >>>> >>>> Thanks for the reply! >>>> >>>> So I am trying to follow your new instructions, but when I am trying to >>>> do >>>> this: >>>> >>>>cd ~/colo-proxy/; git checkout 405527cbfa9f >>>> >>>> I got the following error: >>>> >>>> "error: pathspec '405527cbfa9f' did not match any file(s) known to git." >>>> >>>> I assume it is just a typo? Thank you! >>>> >>>> >>> Hi victor >>> >>> Please git clone >>> >>> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Pating_colo-2Dproxy_tree_changlox&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=pCAkg_8tEQmGEZZoUlyePZjK7z-6aEmp-n6UrQRLWo4&s=Ww-EAIszC-zQuVcDc4XpigwVbMG_4t2SpTg2PV6HTjM&e= >>> *Notice* that, currently we implement colo proxy as a kernel module what >>> is a temporary measure. But further more we'll intergrate it in qemu and >>> drop this one, so both qemu-colo and xen-colo will share the same proxy. >>> Please don't test this colo proxy now, there maybe some bugs, but it's >>> acceptable. >>> >>> Thanks >>> -Xie >>> >>> >>> Victor >>>> >>>> >>> >>> >>> >> >> >> > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] Problem setting up HVM ubuntu guest with Remus
Hi, I was able to follow the instruction on http://wiki.xenproject.org/wiki/Install_Xen_4.2.1_with_Remus_and_DRBD_on_Ubuntu_12.10 to get a working Remus with PV guests. But now I am trying to do the same thing but for HVM guests. So I created a HVM guest on Xen by following https://help.ubuntu.com/community/Xen with the disk location pointing to a logical volume I created name "ubuntu-hvm" under volume group name "gpu2" After that I did "drbdadm create-md ubuntu-hvm" I also edited the /etc/drbd.d/SystemHA_protoD.res so that the disk is pointing to the location where I install and setup the hvm guest. Now I have the drbd connetced between 2 servers. So I edit the .cfg file for the guest U from disk = ['phy:/dev/gpu2/ubuntu-hvm,hda,w','file:/root/ubuntu-12.04.5-server-amd64.iso,hdc:cdrom,r'] to disk = [''drbd:ubuntu-hvm,hda,w''] but when I xm create the guest VM, it told me that "Boot from Hard Disk failed: could not read the boot disk" and short after the HVM guest U is shutdown. Please let me know if you need anymore details. Thank you! Victor ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Patching error while setting up COLO
Hi Changlong, Are you suggesting I should hold off on setting up COLO for now? Thanks! Victor On Thu, Feb 25, 2016 at 8:19 PM, Changlong Xie wrote: > On 02/26/2016 11:38 AM, Yu-An(Victor) Chen wrote: > >> Hi Changlong, >> >> Thanks for the reply! >> >> So I am trying to follow your new instructions, but when I am trying to do >> this: >> >> cd ~/colo-proxy/; git checkout 405527cbfa9f >> >> I got the following error: >> >> "error: pathspec '405527cbfa9f' did not match any file(s) known to git." >> >> I assume it is just a typo? Thank you! >> > > Hi victor > > Please git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Pating_colo-2Dproxy_tree_changlox&d=CwICaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=pCAkg_8tEQmGEZZoUlyePZjK7z-6aEmp-n6UrQRLWo4&s=Ww-EAIszC-zQuVcDc4XpigwVbMG_4t2SpTg2PV6HTjM&e= > *Notice* that, currently we implement colo proxy as a kernel module what > is a temporary measure. But further more we'll intergrate it in qemu and > drop this one, so both qemu-colo and xen-colo will share the same proxy. > Please don't test this colo proxy now, there maybe some bugs, but it's > acceptable. > > Thanks > -Xie > > >> Victor >> > > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] Patching error while setting up COLO
Hi Changlong, Thanks for the reply! So I am trying to follow your new instructions, but when I am trying to do this: cd ~/colo-proxy/; git checkout 405527cbfa9f I got the following error: "error: pathspec '405527cbfa9f' did not match any file(s) known to git." I assume it is just a typo? Thank you! Victor On Thu, Feb 25, 2016 at 2:31 AM, Ian Campbell wrote: > On Thu, 2016-02-25 at 16:49 +0800, Changlong Xie wrote: > > Sorry for this. Since we have no privilege to update the Wiki for a long > > time so most contents of this page are dated : ( > > Anyone can be granted access, it is only a manual process because of > spammers. Please create an account and then fill in the form at: > > https://urldefense.proofpoint.com/v2/url?u=http-3A__xenproject.org_component_content_article_100-2Dmisc_145-2Drequest-2Dto-2Dbe-2Dmade-2Da-2Dwiki-2Deditor.html&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=81rG3g_B23OhGxLWYNA7Hs6cFJy4VxE2Sdqhr0qe7Ds&e= > > > > > Anyway i'll write down detail steps here for colo on XEN. > > > > [Requirements] > > - Hardware > > There is at least one directly connected nic to forward the network > > requests from primary to secondary vm. The directly connected nic must > > not be used by any other purpose. If your guest has more than one nic, > > you should have directly connected nic for each guest nic. If you don't > > have enouth directly connected nic, you can use vlan. > > > > - Dom0 > > 1. Kernel with dom0 support > > 2. kernel module > > nf_conntrack > > nf_conntrack_ipv4 > > nf_nat > > libnl-tools >= 3.0. > > *Note*: If your host os has OEM-released xen tools, *MUST* uninstall it > > first. > > > > - Guest > > Only HVM guest(without pv extensions) is supported now. If you want to > > use OEM released guest os, please use SUSE（we use "SUSE Linux > > Enterprise Server 11" currently）. REDHAT and Ubuntu is not supported > > now because I don't find any way to disable pv extensions. If you want > > to use REDHAT or Ubuntu, you need to build the newest kernel which has > > the parameter xen_nopv. > > > > > > [SETUP] > > - Network link topology > > ref: > https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.xenproject.org_wiki_COLO-5F-2D-5FCoarse-5FGrain-5FLock-5FStepping&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=-vAQ55Sl3YGAa0EmRYKB2pvrRVa_D6nLQCNiOGz7IeI&e= > > > > - Test environment prepare > > > > On both Primary/Secondary hosts: > > 1. > > cd ~ > > git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_colo-2Dproxy&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=uS-WmNmGEeeyytonbMxBCcbSHRzQJXUa6_pvkiGEPh4&e= > > git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_macrosheep_iptables.git&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=ZycjXRwwoyK_M8zU4CS8QsVaLWXzjx8mwtlEEIRlY_I&e= > > git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_torvalds_linux&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=TeB7X5i8Ua7jemThOaxH4oNER5ZMCcs1dyqUxZVIWPY&e= > > git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_qemu-2Dcolo&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=2kSeFd9-e3hISEJ1QZB1uzfVs8TxQwFQ_Rt4EuySfu8&e= > > git clone > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wencongyang_xen&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=ztSkYn_Z-t4_vnCdkHl09RzW8LaD9nnkeebyhac3a_0&e= > > > > 2. Prepare host kernel for Dom0 > > colo-proxy kernel module need cooperate with linux kernel. You should > > patch kernel with ~/colo-proxy/colo-patch-for-kernel.patch > > 1) cd ~/colo-proxy/; git checkout 405527cbfa9f > > 2) cd ~/linux/; git checkout v4.0; git am > > ~/colo-proxy/colo-patch-for-kernel.patch > > 3) cp /boot/config-3.0.76-0.11-xen .config; make menuconfig to config > > your kernel support Dom0. > > Ref: > https://urldefense.proofpoint.com/v2/url?u=http-3A__wiki.xenproject.org_wiki_Mainline-5FLinux-5FKernel-5FConfigs&d=CwIFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=IitX1U91-NhsQt0q4MJOLQ&m=Vnnj3Mx_nu951rfkiPcsuwuboBSLRcYYu30Uho1r7kw&s=3A6be8ZmUe1yTPFuGHZp3fD_LAtyo_5MW2jdVrBSDcs&e= > > 4) make -j8; make modules_install; make install > > 5) reboot > > > > 3. build colo-proxy > > 1) cd ~/colo-proxy/; git checkout 405527cbfa9f; make; make install > > > > 4. build iptables > > 1) cd iptables; ./autogen.sh; ./configure --prefix=/usr/ > > --libdir=/usr/lib64; make; make instal
[Xen-devel] Patching error while setting up COLO
Hi, I am trying to set up COLO by following this http://wiki.xenproject.org/wiki/COLO_-_Coarse_Grain_Lock_Stepping I was able to follow the step up to $git am ~/ColoPatchForQemu/*.patch When I try to run the above command, I got the following error: error: patch failed: include/hw/xen/xen_common.h:186 error: include/hw/xen/xen_common.h: patch does not apply error: patch failed: xen-hvm.c:85 error: xen-hvm.c: patch does not apply Patch failed at 0018 Revert "Xen: Use the ioreq-server API when available" The copy of the patch that failed is found in: /root/xen/tools/qemu-xen-dir/.git/rebase-apply/patch Please let me know how I can solve this patching error. Thank you! Victor ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi all, A little update, so yea when I run with tasks that has utilization less than 0.5. Xen-RTDS is able to complete all those tasks. Thank you guys very much for helping me understand the problem! On Tue, Dec 1, 2015 at 2:11 AM, Lars Kurth wrote: > I wonder whether we need to add some health warnings and recommended > background reading to http://wiki.xenproject.org/wiki/RTDS-Based-Scheduler > Lars > > > On 1 Dec 2015, at 08:59, Dario Faggioli > wrote: > > > > On Sun, 2015-11-29 at 11:44 -0500, Meng Xu wrote: > >> 2015-11-29 11:27 GMT-05:00 Dario Faggioli > >> : > >>> > >>> Mmmm... As I said many times, I don't remember much of all those RT > >>> schedulability formulas, but, is really that simple? > >> > >> Ah, let me clarify... > >> It is not that simple. ;-) I just simplify it, hoping it can simplify > >> the problem and highlight the possible reason. > >> > > Ok, glad to know I haven't completely lost my mind, or anything like > > that! :-) > > > >>> I mean, if the in- > >>> guest scheduling algorithm is global (e.g., global-EDF), the task > >>> could > >>> migrate, couldn't it? > >> > >> Yes. If these partial VCPUs happen to be scheduled "sequentially", > >> the > >> OS inside VM can migrate the task and make the task keep running. But > >> that is not the worst-case for the OS. > >> > > Right, I see it now, and (FWIW) I absolutely agree with the worst-case > > analysis you provided (thanks). I did not get the fact that you were > > talking about the worst-case, sorry for the noise. :-D > > > >> The detailed illustration of the worst case scenario is at Arvind's > >> paper: http://link.springer.com/article/10.1007%2Fs11241-009-9073-x > >> My latest journal paper > >> (http://link.springer.com/article/10.1007%2Fs11241-015-9223-2) > >> tighten > >> the resource supply bound function of the MPR model. I believe the > >> equations are too boring to most of people in the mailing list. > >> > >> So let's avoid the complex equations here. ;-) > >> > > Thanks for this too! :-) > > > > Regards, > > Dario > > -- > > <> (Raistlin Majere) > > - > > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > > > > ___ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Meng and Dario, Thank you both very very much for the explanations! It really help a lot for understanding RT and what I am doing wrong here! I will do more research and reconfigure the experiments. Hope to have some good news soon! Thanks again! On Sun, Nov 29, 2015 at 8:44 AM, Meng Xu wrote: > 2015-11-29 11:27 GMT-05:00 Dario Faggioli : > > On Sun, 2015-11-29 at 10:38 -0500, Meng Xu wrote: > >> > >> > >> 2015-11-29 7:46 GMT-05:00 Yu-An(Victor) Chen : > >> > Hi Meng, > >> > > >> Hi, > >> > >> > > >> > So I will rewrite my setup here again, but this time I shorten the > >> > period and budget for RTDS like you suggested: > >> > > >> Nice! :-) > >> > >> > >> > - > >> > > >> > > >> (I like this line, BTW. :-D) > >> > > >> > > >> > for xen-credit : 2vms (both vm are given 8 vCPUs) sharing 8 cores > >> > (cpu 0-7) using credit scheduler(both with weight of 800 and > >> > capacity of 400) > >> > for xen-rtds: 2 vms (both vm are given 8 vCPUs) sharing 8 cores > >> > (cpu0-7) using RTDS (both with period of 4000(4ms) and budget of > >> > 2000(2ms))) > >> > In both setup, dom0 is using 1 core from cpu 8-15 > >> > > >> > In both setup: > >> > > >> > I loaded VM2 with constant running task with total utilization of 4 > >> > cores. > >> > and in VM1 I run iterations of tasks of total utilization rate of 1 > >> > cores, 2 cores, 3 cores, 4 cores, and then record their > >> > schedulbility. > >> > - > >> > > >> > > >> > So st_jobs_stats for the missed deadline jobs are: > >> > > >> > trial #1 composed of 2 tasks: total tasks utilization rate = 1 > >> > > >> > (period, exe, deadline)=(21ms,12.023ms,21ms) -> miss all deadline > >> > (period, exe, deadline)=(100ms,37.985ms,100ms) -> no miss > >> > > >> yes, this is the information I need and we can solve the mystery > >> now... > >> Let's look at this task: > >> (period, exe, deadline)=(21ms,12.023ms,21ms) -> miss all deadline > >> Its utilization is 12.023 / 21 ~= 0.5614; > >> Your VCPU utilization is only 2ms / 4ms = 0.5 > >> So even when this task is pinned to one VCPU, it will still miss > >> deadline because it has only one thread. :-) > >> > > Mmmm... As I said many times, I don't remember much of all those RT > > schedulability formulas, but, is really that simple? > > Ah, let me clarify... > It is not that simple. ;-) I just simplify it, hoping it can simplify > the problem and highlight the possible reason. > > > I mean, if the in- > > guest scheduling algorithm is global (e.g., global-EDF), the task could > > migrate, couldn't it? > > Yes. If these partial VCPUs happen to be scheduled "sequentially", the > OS inside VM can migrate the task and make the task keep running. But > that is not the worst-case for the OS. > > The worst case for the OS to schedule one task is that all of these > VCPUs are released at the same time, are schedulable as early as > possible in the first period and scheduled as late as possible in the > second period, which will create the largest starvation period to the > VM. > So in the worst case, you won't be able to schedule a task with > utilization U on k VCPUs with utilzation U, no matter how large k is. > (Think about that these k VCPUs are always scheduled at the same > time.) > > The detailed illustration of the worst case scenario is at Arvind's > paper: http://link.springer.com/article/10.1007%2Fs11241-009-9073-x > My latest journal paper > (http://link.springer.com/article/10.1007%2Fs11241-015-9223-2) tighten > the resource supply bound function of the MPR model. I believe the > equations are too boring to most of people in the mailing list. > > So let's avoid the complex equations here. ;-) > > To Yu-An, > The basic idea is, as Dario mentioned in the previous email, that the > configuration of VCPUs are important to provide the schedulability of > tasks inside VM. Especially, if you are doing research in RT, you need > to (maybe have to) know the RT scheduling theory. :-) > > > >
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Meng, So I will rewrite my setup here again, but this time I shorten the period and budget for RTDS like you suggested: - for xen-credit : 2vms (both vm are given 8 vCPUs) sharing 8 cores (cpu 0-7) using credit scheduler(both with weight of 800 and capacity of 400) for xen-rtds: 2 vms (both vm are given 8 vCPUs) sharing 8 cores (cpu0-7) using RTDS (both with period of 4000(4ms) and budget of 2000(2ms))) In both setup, dom0 is using 1 core from cpu 8-15 In both setup: I loaded VM2 with constant running task with total utilization of 4 cores. and in VM1 I run iterations of tasks of total utilization rate of 1 cores, 2 cores, 3 cores, 4 cores, and then record their schedulbility. - So st_jobs_stats for the missed deadline jobs are: trial #1 composed of 2 tasks: total tasks utilization rate = 1 (period, exe, deadline)=(21ms,12.023ms,21ms) -> miss all deadline (period, exe, deadline)=(100ms,37.985ms,100ms) -> no miss trial #2 composed of 2 tasks: total tasks utilization rate = 1 (period, exe, deadline)=(68ms,40.685ms,68ms) -> miss all deadline (period, exe, deadline)=(70ms,28.118ms,70ms) -> no miss trial #3 composed of 2 tasks: total tasks utilization rate = 1 (period, exe, deadline)=(16ms,11.613ms,16ms) -> miss all deadline (period, exe, deadline)=(46ms,12.612ms,46ms) -> no miss I do notice that for within the task that misses deadline, the completion time get progressively longer, for example: for trial #3, a snapshot of the task st_jobs_stats tells me that the completion time of the job is 79ms and then 87ms, and then 95ms ok, I will look into the reference you provide, I just started my research in the rt fields, there is still a lot for me to learn. Thank you again for reply! Thank you! On Sat, Nov 28, 2015 at 7:09 AM, Meng Xu wrote: > 2015-11-28 7:20 GMT-05:00 Yu-An(Victor) Chen : > > Hi Meng, > > > > Thank you so much for being this patience. > > > > So a task set is composed of a collection of real-time tasks, and each > > real-time task is a sequence of jobs that are released periodically... > All > > jobs are periodic, where each job Ti is defined by a period (and > deadline) > > pi and a worse-case execution time ei, with pi ≥ ei ≥ 0 and pi, ei ∈ > > integers. Each job is comprised of a number of iterations of floating > point > > operations during each job. This is based on the base task.c provided > with > > the LITMUSRT userspace library. > > I knew this information. Basically, I did the experiment with this > kind of configuration before. What I want to know is what is the range > of period, execution and deadline you have for the taskset and when a > taskset is unschedulable under your experiment, what is the > st_jobs_stats result from the LITMUS which will show which job of > which task misses deadline. If you try to service a task with period = > 100, exe = 50, deadline = 100, specificed as (100, 50, 100), with a > VCPU with period = 100 and budget = 100, you will never schedule this > task. The reason is because the VCPU can be unavailable when task is > released. > A theoretical analysis can be found in this paper "Periodic Resource > Model for Compositional RealTime Guarantees" > > http://repository.upenn.edu/cgi/viewcontent.cgi?article=1033&context=cis_reports > > BTW, I'm assuming you are familiar with the schedulability test for > real time systems since you are talking about schedulability. If my > assumption is wrong, you should have a look at the survey paper A > survey of hard real-time scheduling for multiprocessor systems > (http://dl.acm.org/citation.cfm?id=1978814). This will at least tell > you when you should expect a taskset is schedulable in theory. If a > taskset is claimed unschedulabled in theory, it is very likely > unscheduble in practice. In addition, it will tell you taskset > utilization is not the only factor that affects the schedulablity. > Other factors, such as the highest taks utilization in a taskset, > scheduling algorithm, task period relation, can affect schedulability > also. > > Best, > > Meng > > > > > > > > On Fri, Nov 27, 2015 at 4:17 PM, Meng Xu wrote: > >> > >> 2015-11-27 14:50 GMT-05:00 Yu-An(Victor) Chen : > >> > Hi Dario & Meng, > >> > > >> > Thanks for your analysis! > >> > > >> > VM1 and VM2 both are given 8 vCPUs and sharing physical CPU 0-7. So in > >> > theory,"VM1 can get the services of 400%" > >> > And yes, Dario, your expla
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Meng, Thank you so much for being this patience. So a task set is composed of a collection of real-time tasks, and each real-time task is a sequence of jobs that are released periodically... All jobs are periodic, where each job Ti is defined by a period (and deadline) pi and a worse-case execution time ei, with pi ≥ ei ≥ 0 and pi, ei ∈ integers. Each job is comprised of a number of iterations of floating point operations during each job. This is based on the base task.c provided with the LITMUSRT userspace library. So in a tasks set, there are many tasks, and in each tasks, there are a sequence of jobs, if any job misses deadline, then the whole taskset is qualified as failed to be schedulable. Thank you Victor On Fri, Nov 27, 2015 at 4:17 PM, Meng Xu wrote: > 2015-11-27 14:50 GMT-05:00 Yu-An(Victor) Chen : > > Hi Dario & Meng, > > > > Thanks for your analysis! > > > > VM1 and VM2 both are given 8 vCPUs and sharing physical CPU 0-7. So in > > theory,"VM1 can get the services of 400%" > > And yes, Dario, your explanation about the task utilization is correct. > > > > So the resource configuration as I mentioned before is: > > > > for xen-credit : 2vms (both vm are given 8 vCPUs) sharing 8 cores (cpu > 0-7) > > using credit scheduler(both with weight of 800 and capacity of 400) > > for xen-rtds: 2 vms (both vm are given 8 vCPUs) sharing 8 cores (cpu0-7) > > using RTDS (both with period of 1 and budget of 5000) > > In both setup, dom0 is using 1 core from cpu 8-15 > > > > In both setup: > > > > I loaded VM2 with constant running task with total utilization of 4 > cores. > > and in VM1 I run iterations of tasks of total utilization rate of 1 > cores, 2 > > cores, 3 cores, 4 cores, and then record their schedulbility. > > > > Attached is the result plot. > > > > > > I have tried with the newest litmust-rt, and rtxen is still performing > > poorly. > > What is the characteristics of tasks you generated? When a taskset > miss ddl., which task inside miss deadline? > > Meng > > > > > > Thank you both very much again, if there is any unclear part, please > lemme > > know, thx! > > > > Victor > > > > > > > > On Fri, Nov 27, 2015 at 9:41 AM, Meng Xu wrote: > >> > >> 2015-11-27 12:23 GMT-05:00 Dario Faggioli : > >> > On Fri, 2015-11-27 at 08:36 -0800, Yu-An(Victor) Chen wrote: > >> >> Hi Dario, > >> >> > >> > Hi, > >> > > >> >> Thanks for the reply! > >> >> > >> > You're welcome. :-) > >> > > >> > I'm adding Meng to Cc... > >> > > >> > >> Thanks! :-) > >> > >> >> My goal for the experiment is to show that xen rtds scheduler is > >> >> better than credit scheduler when it comes to real time tasks. > >> >> so my set up is: > >> >> > >> >> for xen-credit : 2vms sharing 8 cores (cpu 0-7) using credit > >> >> scheduler(both with weight of 800 and capacity of 400) > >> > >> So you set up 400% cpu cap for each VM. In other words, each VM will > >> have computation capacity almost equal to 4 cores. Because VCPUs are > >> also scheduled, the four-core capacity is not equal to 4 physical core > >> in bare metal, because the resource supplied to tasks from VCPUs also > >> depend on the scheduling pattern (which affect the resource supply > >> pattern) of the VCPUs. > >> > >> >> for xen-rtds: 2 vms sharing 8 cores (cpu0-7) using RTDS (both with > >> >> period of 1 and budget of 5000) > >> > >> How many VCPUs for each VM? If each VM has 4 VCPU, each VM has only > >> 200% CPU capacity, which is only half compared to the configuration > >> you made for credit scheduler. > >> > >> >> in both setup, dom0 is using 1 core from cpu 8-15 > >> > >> Do you have some quick evaluation report (similar to the evaluation > >> section in academic papers) that describe how you did the experiments, > >> so that we can have a better guess on where goes wrong. > >> > >> Right now, I'm guessing that: the resource configured for each VM > >> under credit and rtds schedulers are not the same, and it is possible > >> that some parameters are not configured correctly. > >> > >> Another thing is that: > >> credit scheduler is work conserving, while RTDS is not. > >> So under the under-loaded situation, you will see credit scheduler may > >> work better because it try to use as much resource as it could. You > >> can make the comparision more failrly by setting the cap for credit > >> scheduler as you did, and running some background VM or tasks to > >> consume the idle resource. > >> > >> Meng > > > > > > > > -- > > > --- > Meng Xu > PhD Student in Computer and Information Science > University of Pennsylvania > http://www.cis.upenn.edu/~mengxu/ > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Dario & Meng, Thanks for your analysis! VM1 and VM2 both are given 8 vCPUs and sharing physical CPU 0-7. So in theory,"VM1 can get the services of 400%" And yes, Dario, your explanation about the task utilization is correct. So the resource configuration as I mentioned before is: for xen-credit : 2vms (both vm are given 8 vCPUs) sharing 8 cores (cpu 0-7) using credit scheduler(both with weight of 800 and capacity of 400) for xen-rtds: 2 vms (both vm are given 8 vCPUs) sharing 8 cores (cpu0-7) using RTDS (both with period of 1 and budget of 5000) In both setup, dom0 is using 1 core from cpu 8-15 In both setup: I loaded VM2 with constant running task with total utilization of 4 cores. and in VM1 I run iterations of tasks of total utilization rate of 1 cores, 2 cores, 3 cores, 4 cores, and then record their schedulbility. Attached is the result plot. I have tried with the newest litmust-rt, and rtxen is still performing poorly. Thank you both very much again, if there is any unclear part, please lemme know, thx! Victor On Fri, Nov 27, 2015 at 9:41 AM, Meng Xu wrote: > 2015-11-27 12:23 GMT-05:00 Dario Faggioli : > > On Fri, 2015-11-27 at 08:36 -0800, Yu-An(Victor) Chen wrote: > >> Hi Dario, > >> > > Hi, > > > >> Thanks for the reply! > >> > > You're welcome. :-) > > > > I'm adding Meng to Cc... > > > > Thanks! :-) > > >> My goal for the experiment is to show that xen rtds scheduler is > >> better than credit scheduler when it comes to real time tasks. > >> so my set up is: > >> > >> for xen-credit : 2vms sharing 8 cores (cpu 0-7) using credit > >> scheduler(both with weight of 800 and capacity of 400) > > So you set up 400% cpu cap for each VM. In other words, each VM will > have computation capacity almost equal to 4 cores. Because VCPUs are > also scheduled, the four-core capacity is not equal to 4 physical core > in bare metal, because the resource supplied to tasks from VCPUs also > depend on the scheduling pattern (which affect the resource supply > pattern) of the VCPUs. > > >> for xen-rtds: 2 vms sharing 8 cores (cpu0-7) using RTDS (both with > >> period of 1 and budget of 5000) > > How many VCPUs for each VM? If each VM has 4 VCPU, each VM has only > 200% CPU capacity, which is only half compared to the configuration > you made for credit scheduler. > > >> in both setup, dom0 is using 1 core from cpu 8-15 > > Do you have some quick evaluation report (similar to the evaluation > section in academic papers) that describe how you did the experiments, > so that we can have a better guess on where goes wrong. > > Right now, I'm guessing that: the resource configured for each VM > under credit and rtds schedulers are not the same, and it is possible > that some parameters are not configured correctly. > > Another thing is that: > credit scheduler is work conserving, while RTDS is not. > So under the under-loaded situation, you will see credit scheduler may > work better because it try to use as much resource as it could. You > can make the comparision more failrly by setting the cap for credit > scheduler as you did, and running some background VM or tasks to > consume the idle resource. > > Meng > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Dario, Thanks for the reply! My goal for the experiment is to show that xen rtds scheduler is better than credit scheduler when it comes to real time tasks. so my set up is: for xen-credit : 2vms sharing 8 cores (cpu 0-7) using credit scheduler(both with weight of 800 and capacity of 400) for xen-rtds: 2 vms sharing 8 cores (cpu0-7) using RTDS (both with period of 1 and budget of 5000) in both setup, dom0 is using 1 core from cpu 8-15 VM2 will run tasks that has utilization of 4 cores. VM1 will run tasks that has utilization from 1 to 4, and I will record VM1's schedulbility. I am hoping to see that the VM1 using xen-rtds will perform better than the VM1 using xen-credit, but what I am seeing is the opposite. Thank you! Victor On Tue, Nov 24, 2015 at 4:15 PM, Dario Faggioli wrote: > On Mon, 2015-11-23 at 07:42 -0800, Yu-An(Victor) Chen wrote: > > Hi all, > > > Hello, > > > So I was doing some experiments to evaluate RTDS scheduler > > schedubility of real time tasks using 1vm with period of 1 and > > budget of 1. The experiment results turn out as expected(perform > > better than xen-credit). > > > > But when I tried to perform similar experiments with 2 vms (both with > > now period of 1 and budget of 5000). The schedubility of real > > time tasks turn out really bad. Even if I have one vm idling and the > > other vm running the real time tasks, the schedubility of that vm is > > still really poor(worse than xen-credit). Am I missing some > > configuration I should have set for 2vm cases? Thank you > > > What is it that you are trying to prove with this setup? This is > despite all Meng is already saying about the non-work conserving nature > of RTDS, and about the LITMUS IPI bug. > > In fact, in general, real-time schedulers are really good at isolating > workloads, with precise time guarantees. If you have stuff that needs > to be done in 2 VMs, and you use RTDS for scheduling the 2 VMs, you'll > get good and precisely characterized isolation between them. > > But if you put all the stuff in only 1 VM, and then limit its own > utilization, all you are doing is making it hard for the things inside > the VM itself to achieve their target performance, with respect to both > an instance of RTDS where that VM has 100% utilization, as well as with > (almost) any general purpose scheduler. > > Then, again, as Meng is saying, if you not only have "stuff" to do > inside the VM, but you are interested in in-guest real-time, then the > scheduling parameters of the VM(s) and the ones of the tasks in the > guest(s), should be set according to a proper real-time hierarchical > scheduling scheme that allows for guarantees to be met. > > Regards, > Dario > -- > <> (Raistlin Majere) > - > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Meng, Thank you again for your kindly reply! I am currently using "litmus-rt-2014.2.patch" and with my labmates' patch for IPI interrupt (https://github.com/LITMUS-RT/liblitmus/pull/1/files) I asked my labmate about it, he said he was not sure if there is more IPI interrupt bug other than then one he fixed. I can am using "st_trace" to check the success rate of the tasks. it only gives me the how much of the deadline I am missing for each job. which TRACE function in LITMUS, do you mean? litmus_log?ft_cpu_traceX?ft_msg_traceX?sched_trace? or st_trace? https://wiki.litmus-rt.org/litmus/Tracing At this point I think the best way is to install the newest litmus-rt (litmus-rt-2015.1.patch) ? Again, thank you very much, I really appreciate your help! Victor On Mon, Nov 23, 2015 at 6:23 PM, Meng Xu wrote: > Hi, > > 2015-11-23 11:35 GMT-05:00 Yu-An(Victor) Chen : > > Hi Meng, > > > > Thank you very much for replying! > > > > The RT tasks I am running for each trial at a certain utilization rate > is a > > collection of real-time tasks, and each real-time task is a sequence of > jobs > > that are released periodically. All jobs are periodic, where each job is > > defined by a period (and deadline) and a worse-case execution time. Each > job > > is just running of a number of iterations of floating point operations. > This > > is based on the base task.c provided with the LITMUSRT userspace > library. So > > Yes they are independent and not I/O or memory intensive. > > Ah, I see. Which version of LITMUSRT did you use? LITMUS^RT has a bug > in Xen environment. The IPI interrupt is not handled properly in > LITMUS on Xen. With the bug, the system performance is worse than > when the system is loaded, because LITMUS scheduler just fails to > respond due to the IPI ignorance. > > IIRC, this bug was fixed in the latest LITMUSRT code. > Did you use the latest LITMUS code? > > One way to debug the issue is: > Can you enable the TRACE in LITMUS and collact the scheduling log in > the scenario when you see the bad performance? > > > > > The period distribution I have for each task is (10ms,100ms) which is > bigger > > than the VM period I specified using xl sched-rtds (1us), but I guess > > the lower bound is too close to the VM period. So by your suggestion, > maybe > > shorten the period for VM can improve the performance? > > Yes. At least it shorten the starvation interval. > > Meng > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi Meng, Thank you very much for replying! The RT tasks I am running for each trial at a certain utilization rate is a collection of real-time tasks, and each real-time task is a sequence of jobs that are released periodically. All jobs are periodic, where each job is defined by a period (and deadline) and a worse-case execution time. Each job is just running of a number of iterations of floating point operations. This is based on the base task.c provided with the LITMUSRT userspace library. So Yes they are independent and not I/O or memory intensive. The period distribution I have for each task is (10ms,100ms) which is bigger than the VM period I specified using xl sched-rtds (1us), but I guess the lower bound is too close to the VM period. So by your suggestion, maybe shorten the period for VM can improve the performance? Thank you! Victor On Mon, Nov 23, 2015 at 8:07 AM, Meng Xu wrote: > Hi Yu-An, > > 2015-11-23 10:42 GMT-05:00 Yu-An(Victor) Chen : > > > > Hi all, > > > > So I was doing some experiments to evaluate RTDS scheduler schedubility > of real time tasks using 1vm with period of 1 and budget of 1. The > experiment results turn out as expected(perform better than xen-credit). > > > Thank you very much for doing this test and trying RTDS scheduler. :-) > > > > > > > But when I tried to perform similar experiments with 2 vms (both with > now period of 1 and budget of 5000). The schedubility of real time > tasks turn out really bad. Even if I have one vm idling and the other vm > running the real time tasks, the schedubility of that vm is still really > poor(worse than xen-credit). > > > RTDS scheduler is not work-conserving, while credit scheduler is. Even > if you keep one VM idling, the real-time VM that you run the RT tasks > will still get 5000 budget in each 1. This is what the budget > replenish policy requires. However, credit scheduler will try to use > all budget (in its best effort) when there is no others running. > > Another factor that may affect the performance is the period of the > VM. The VM period should be smaller than the tasks' period inside. > Keep in mind that in the worst case, the VM may not get resource for > 2*(period - budget) time, when the VM's first VCPU get budget at the > beginning of the first period but get budget at the end of the next > period. > > > > > > Am I missing some configuration I should have set for 2vm cases? Thank > you > > > > my setup is the following: > > Using Xen 4.5.0. > > 2vm sharing core 0-7, with RTDS scheduler, both has period of 1 and > budget of 5000 > > Dom0 using one core from CPU 8-15, with RTDS scheduler, period of 1 > and budget of 1 > > The experiment I am doing is running real time tasks with different > total utilization rate and measure the task success rate in the vm. > > > Which kind of RT tasks are you running inside VM? Are they > independent? Are they involving a lot of I/O or memory? > > Thanks, > > Meng > > > > > -- > > > --- > Meng Xu > PhD Student in Computer and Information Science > University of Pennsylvania > http://www.cis.upenn.edu/~mengxu/ > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] xen 4.5.0 rtds scheduler perform poorly with 2vms
Hi all, So I was doing some experiments to evaluate RTDS scheduler schedubility of real time tasks using 1vm with period of 1 and budget of 1. The experiment results turn out as expected(perform better than xen-credit). But when I tried to perform similar experiments with 2 vms (both with now period of 1 and budget of 5000). The schedubility of real time tasks turn out really bad. Even if I have one vm idling and the other vm running the real time tasks, the schedubility of that vm is still really poor(worse than xen-credit). Am I missing some configuration I should have set for 2vm cases? Thank you my setup is the following: Using Xen 4.5.0. 2vm sharing core 0-7, with RTDS scheduler, both has period of 1 and budget of 5000 Dom0 using one core from CPU 8-15, with RTDS scheduler, period of 1 and budget of 1 The experiment I am doing is running real time tasks with different total utilization rate and measure the task success rate in the vm. Please let me know if you need any more information. Thank you! Victor ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] how to trace scheduling overheads using xentrace/xenalyze
Hi Dario,
Thank you very much for your reply!!! It is very nice of you :)
Yes I have checked the blog entry and that is actually where I started!
To define my "scheduling overhead", I think I should tell you what I am
trying to accomplish:
So I am currently using Xen 4.5 to help us understand more about real time
virtualization. When I am experimenting with Xen-4.5, I observed that the
guest VM tend to perform not as well when there are a lot of small jobs in
a given period.
I think the reason behind it is because even though the jobs are small, but
there are so many tasks so the large amount of scheduling overhead for each
job cause the performance to get worse.
So I want to measure the scheduling overhead for the tasks.
So I guess my definition for "scheduling overhead" in this case will be the
time interval between when the clock interrupts for scheduling before a job
is run and when the job starts.
Basically, I want to see how long does it take to schedule a job and how
long does it take to actually run the job.
So in that case just looking at how vcpus are scheduled is not enough? If
you think this concept is incorrect, please correct me, thx!
Oh and yes I can get a similar output file by using xentrace -D -e
0x0002f000 trace.bin & xenalyze dump-all.
that brings me to 2 more questions about how to read this dump-all output:
First Question: how do I read those lines, for example :
0.19969 x-- -- d32767v0 22805(2:2:805) 5 [ 7fff 0 0 0 0 ]
I can only tell that the first number( 0.19969) means time, and
d32767v0 means "domain 32767 at vcpu0".
what does 22805(2:2:805) 5 [ 7fff 0 0 0 0 ] each stands for? hopefully it
contain the information I need.
Second Question: Why are there domain 32768 and 32767? I never created
those domains and they took up most of the dump-all file.
Again thank you very much for your reply. and sorry for the long replies
if my reply format is wrong, I am new to mailing list too.
Victor
On Wed, Oct 7, 2015 at 3:32 AM, Dario Faggioli wrote:
> On Tue, 2015-10-06 at 20:46 -0700, Yu-An(Victor) Chen wrote:
> > Hi,
> >
> Hi,
>
> > I am new to xen environment
> >
> Welcome :-)
>
> > and I am wondering how to trace scheduling overhead of guest vm using
> > xentrace/xenalyze ?
> >
> Have you had a look at this blog entry? It's from some time ago, but
> the basic concepts should still hold.
>
> https://blog.xenproject.org/2012/09/27/tracing-with-xentrace-and-xenal
> yze/
>
> Of course, that tells how to use the tools (xentrace and xenalyze), not
> how to "trace scheduling overhead". For doing so, I think you should
> first define what it is really that you want to measure (many
> definitions of scheduling overhead are possible) and, only after that,
> check whether it is possible to do that with existing tools.
>
> > I have tried using $xentrace -D -e all -S 256 {filename}
> >
> > and then use various xenalyze options but most of them gave me empty
> > result, and I dont really see where I can get scheduling overhead so
>
> >
> Mmm... The command above works for me (just tried). However, '-e all'
> is a lot of data, and it may actually take a bit to xenalyze to parse
> it.
>
> Maybe, if you are interested in tracing scheduling related events, use
> the appropriate event mask?
>
> > I can see how are the jobs are scheduled and its execution time and
> > stuff. Please point me to a direction. Thank you!
> >
> Again, you should detail more what you think 'scheduling overhead' is.
> If you are interested in seeing how and where the vcpus are scheduled,
> you want a dump.
>
> With this:
>
> xentrace -D -e 0x0002f000 trace.bin
>
> and then this:
>
> xenalyze --dump-all trace.bin
>
> Here's an excerpt of what I get:
>
> ] 0.19647 x-- -- d32767v0 22802(2:2:802) 0 [ ]
> ] 0.19969 x-- -- d32767v0 22805(2:2:805) 5 [ 7fff 0 0 0 0 ]
>0.20474 x-- -- d32767v0 runstate_continue d32767v0
> running->running
> ] 0.21170 x-- -- d32767v4 22802(2:2:802) 0 [ ]
> ] 0.21370 x-- -- d32767v4 22805(2:2:805) 5 [ 47fff 0 0 0 0
> ]
>0.21817 x-- -- d32767v4 runstate_continue d32767v4
> running->running
> ] 0.22235 -x- -- d32767v9 22802(2:2:802) 0 [ ]
> ] 0.22467 -x- -- d32767v9 22805(2:2:805) 5 [ 97fff 0 0 0 0
> ]
>0.22983 -x- -- d32767v9 runstate_continue d32767v9
> running->running
> ] 0.23438 x-- -- d32767v8 22802(2:2:802) 0 [ ]
> ] 0.23638 x-- -- d32767v8 22805(2:2:805) 5 [ 87fff 0 0 0 0
> ]
>
> Regards,
> Dario
> --
> <> (Raistlin Majere)
> -
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
>
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
[Xen-devel] how to use xentrace_format correctly
Hi, I am not sure how to use xentrace_format given that I have a trace file. Can someone gave me an example def file and the correct commands?? Thank you! ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] how to trace scheduling overheads using xentrace/xenalyze
Hi,
I am new to xen environment and I am wondering how to trace scheduling
overhead of guest vm using xentrace/xenalyze ?
I have tried using $xentrace -D -e all -S 256 {filename}
and then use various xenalyze options but most of them gave me empty
result, and I dont really see where I can get scheduling overhead so I can
see how are the jobs are scheduled and its execution time and stuff. Please
point me to a direction. Thank you!
Victor
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
