Sure,
To reproduce the problem using ovs 2.6.1 tarball from the official site. Download, extract and access the source directory, than:
1) ./boot.sh
2) ./configure --with-dpdk
3) export LD_AS_NEEDED=1 # This is the command that inserts the issue
4) make
5) ./ovsdb/ovsdb-server --remote=punix:/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \
--pidfile --detach
6) export DB_SOCK=/var/run/openvswitch/db.sock
7) ./vswitchd/ovs-vswitchd unix:$DB_SOCK --pidfile --detach
Similarly, I was able to remove the issue from Clear Linux openvswitch SPRM including the line "unset LD_AS_NEEDED" in the spec file, just before the make command, than recreating and installing the modified package.
Regards,
Marcos Schwarz
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>
Cc: "dev" <dev(a)lists.clearlinux.org>
Sent: Monday, January 30, 2017 6:35:25 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
Perfect, thanks I going to work on this, and I hope to have news soon, btw could you tell me
The steps that you are running in order to reproduce exactly the problem.
Regards.
Mario Alfredo Carrillo Arevalo
> -----Original Message-----
> From: Marcos Felipe Schwarz [mailto:marcos.schwarz(a)rnp.br]
> Sent: Monday, January 30, 2017 2:28 PM
> To: Arevalo, Mario Alfredo C <mario.alfredo.c.arevalo(a)intel.com>
> Cc: dev <dev(a)lists.clearlinux.org>
> Subject: Re: DPDK support is openvswitch is broken and configuration
> documentation is outdated
>
> I use either no path, or the default path which gives the same result
> (/usr/share/dpdk).
>
> Just confirmed that unseting LD_AS_NEEDED on the openvswitch.spec just
> before the "make" command makes ovs work.
> So it is probably a linking issue... But I don't know what the proper solution
> should be.
>
> Regards,
>
> Marcos Schwarz
>
> ----- Original Message -----
> From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
> To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>
> Cc: "dev" <dev(a)lists.clearlinux.org>
> Sent: Monday, January 30, 2017 6:20:38 PM
> Subject: RE: DPDK support is openvswitch is broken and configuration
> documentation is outdated
>
> One question more, could you tell me which path do you use with the option
> "--with-dpdk" in your manual build?
>
> Regards.
> Mario Alfredo Carrillo Arevalo
> ________________________________________
> From: Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
> Sent: Monday, January 30, 2017 12:07 PM
> To: Arevalo, Mario Alfredo C
> Cc: dev
> Subject: Re: DPDK support is openvswitch is broken and configuration
> documentation is outdated
>
> I could reproduce a non working manual compilation forcing LD_AS_NEEDED.
>
> ----- Original Message -----
> From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
> To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "dev"
> <dev(a)lists.clearlinux.org>
> Sent: Monday, January 30, 2017 4:01:05 PM
> Subject: RE: DPDK support is openvswitch is broken and configuration
> documentation is outdated
>
> Hi Marcos Schwarz, which configuration do you use when you build OVS
> manually ?
>
> Regards.
> Mario Alfredo Carrillo Arevalo.
>
> ________________________________________
> From: Dev [dev-bounces(a)lists.clearlinux.org] on behalf of Marcos Felipe
> Schwarz [marcos.schwarz(a)rnp.br]
> Sent: Monday, January 30, 2017 8:21 AM
> To: dev
> Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and
> configuration documentation is outdated
>
> Thanks Arjan,
>
> That made OVS work on the Rangeley machine, but I'm reaching onto a
> another problem.
> I can't make the openvswitch that comes with clearlinux work with physical
> NICs, it never detects the dpdk0 device.
> I manually compiled openvswitch source on the same and it works as
> expected. After some testes I isolated the problem to the ovs-vswitchd
> binary execution.
> If I use all stock openvswitch files on clearlinux combined with the manually
> compiled ovs-vswitchd it all works.
> I've attached both logs, from all stock clr files (ovs-clr.log) and using stock clr
> files + ovs-vswitchd manually compiled (ovs-orig.log), which shows that the
> interfaces are not detected on clr ovs-vswitchd.
>
> Do you have any suggestion on where should I look further for solving this
> issue?
> PS: I made testes using only uio_pci_generic and igb_uio modules.
>
> Regards,
>
> Marcos Scwharz
>
> ----- Original Message -----
> From: "Arjan van de Ven" <arjan(a)linux.intel.com>
> To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "Arevalo, Mario
> Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
> Cc: "dev" <dev(a)lists.clearlinux.org>
> Sent: Friday, January 13, 2017 1:07:53 PM
> Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and
> configuration documentation is outdated
>
> On 1/13/2017 6:55 AM, Arjan van de Ven wrote:
> > On 1/13/2017 6:41 AM, Marcos Felipe Schwarz wrote:
> >> Further looking into the problem, the illegal instruction is shrx from bmi2
> instruction set and this CPU don't support it.
> >> So this CPU is not supported by Clear Linux by deafult?
> >>
> >
> >
> > so the dpdk makefiles sneak in an -march=native and our build servers are
> supporting BMI2
> >
> > will fix
> >
>
> fixed for the afternoon build today.
I use either no path, or the default path which gives the same result (/usr/share/dpdk).
Just confirmed that unseting LD_AS_NEEDED on the openvswitch.spec just before the "make" command makes ovs work.
So it is probably a linking issue... But I don't know what the proper solution should be.
Regards,
Marcos Schwarz
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>
Cc: "dev" <dev(a)lists.clearlinux.org>
Sent: Monday, January 30, 2017 6:20:38 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
One question more, could you tell me which path do you use with the option "--with-dpdk" in your manual build?
Regards.
Mario Alfredo Carrillo Arevalo
________________________________________
From: Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
Sent: Monday, January 30, 2017 12:07 PM
To: Arevalo, Mario Alfredo C
Cc: dev
Subject: Re: DPDK support is openvswitch is broken and configuration documentation is outdated
I could reproduce a non working manual compilation forcing LD_AS_NEEDED.
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "dev" <dev(a)lists.clearlinux.org>
Sent: Monday, January 30, 2017 4:01:05 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
Hi Marcos Schwarz, which configuration do you use when you build OVS manually ?
Regards.
Mario Alfredo Carrillo Arevalo.
________________________________________
From: Dev [dev-bounces(a)lists.clearlinux.org] on behalf of Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
Sent: Monday, January 30, 2017 8:21 AM
To: dev
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
Thanks Arjan,
That made OVS work on the Rangeley machine, but I'm reaching onto a another problem.
I can't make the openvswitch that comes with clearlinux work with physical NICs, it never detects the dpdk0 device.
I manually compiled openvswitch source on the same and it works as expected. After some testes I isolated the problem to the ovs-vswitchd binary execution.
If I use all stock openvswitch files on clearlinux combined with the manually compiled ovs-vswitchd it all works.
I've attached both logs, from all stock clr files (ovs-clr.log) and using stock clr files + ovs-vswitchd manually compiled (ovs-orig.log), which shows that the interfaces are not detected on clr ovs-vswitchd.
Do you have any suggestion on where should I look further for solving this issue?
PS: I made testes using only uio_pci_generic and igb_uio modules.
Regards,
Marcos Scwharz
----- Original Message -----
From: "Arjan van de Ven" <arjan(a)linux.intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
Cc: "dev" <dev(a)lists.clearlinux.org>
Sent: Friday, January 13, 2017 1:07:53 PM
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
On 1/13/2017 6:55 AM, Arjan van de Ven wrote:
> On 1/13/2017 6:41 AM, Marcos Felipe Schwarz wrote:
>> Further looking into the problem, the illegal instruction is shrx from bmi2 instruction set and this CPU don't support it.
>> So this CPU is not supported by Clear Linux by deafult?
>>
>
>
> so the dpdk makefiles sneak in an -march=native and our build servers are supporting BMI2
>
> will fix
>
fixed for the afternoon build today.
I could reproduce a non working manual compilation forcing LD_AS_NEEDED.
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "dev" <dev(a)lists.clearlinux.org>
Sent: Monday, January 30, 2017 4:01:05 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
Hi Marcos Schwarz, which configuration do you use when you build OVS manually ?
Regards.
Mario Alfredo Carrillo Arevalo.
________________________________________
From: Dev [dev-bounces(a)lists.clearlinux.org] on behalf of Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
Sent: Monday, January 30, 2017 8:21 AM
To: dev
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
Thanks Arjan,
That made OVS work on the Rangeley machine, but I'm reaching onto a another problem.
I can't make the openvswitch that comes with clearlinux work with physical NICs, it never detects the dpdk0 device.
I manually compiled openvswitch source on the same and it works as expected. After some testes I isolated the problem to the ovs-vswitchd binary execution.
If I use all stock openvswitch files on clearlinux combined with the manually compiled ovs-vswitchd it all works.
I've attached both logs, from all stock clr files (ovs-clr.log) and using stock clr files + ovs-vswitchd manually compiled (ovs-orig.log), which shows that the interfaces are not detected on clr ovs-vswitchd.
Do you have any suggestion on where should I look further for solving this issue?
PS: I made testes using only uio_pci_generic and igb_uio modules.
Regards,
Marcos Scwharz
----- Original Message -----
From: "Arjan van de Ven" <arjan(a)linux.intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
Cc: "dev" <dev(a)lists.clearlinux.org>
Sent: Friday, January 13, 2017 1:07:53 PM
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
On 1/13/2017 6:55 AM, Arjan van de Ven wrote:
> On 1/13/2017 6:41 AM, Marcos Felipe Schwarz wrote:
>> Further looking into the problem, the illegal instruction is shrx from bmi2 instruction set and this CPU don't support it.
>> So this CPU is not supported by Clear Linux by deafult?
>>
>
>
> so the dpdk makefiles sneak in an -march=native and our build servers are supporting BMI2
>
> will fix
>
fixed for the afternoon build today.
I used just the --with-dpdk configuration.
Regards,
Marcos Schwarz
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "dev" <dev(a)lists.clearlinux.org>
Sent: Monday, January 30, 2017 4:01:05 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
Hi Marcos Schwarz, which configuration do you use when you build OVS manually ?
Regards.
Mario Alfredo Carrillo Arevalo.
________________________________________
From: Dev [dev-bounces(a)lists.clearlinux.org] on behalf of Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
Sent: Monday, January 30, 2017 8:21 AM
To: dev
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
Thanks Arjan,
That made OVS work on the Rangeley machine, but I'm reaching onto a another problem.
I can't make the openvswitch that comes with clearlinux work with physical NICs, it never detects the dpdk0 device.
I manually compiled openvswitch source on the same and it works as expected. After some testes I isolated the problem to the ovs-vswitchd binary execution.
If I use all stock openvswitch files on clearlinux combined with the manually compiled ovs-vswitchd it all works.
I've attached both logs, from all stock clr files (ovs-clr.log) and using stock clr files + ovs-vswitchd manually compiled (ovs-orig.log), which shows that the interfaces are not detected on clr ovs-vswitchd.
Do you have any suggestion on where should I look further for solving this issue?
PS: I made testes using only uio_pci_generic and igb_uio modules.
Regards,
Marcos Scwharz
----- Original Message -----
From: "Arjan van de Ven" <arjan(a)linux.intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
Cc: "dev" <dev(a)lists.clearlinux.org>
Sent: Friday, January 13, 2017 1:07:53 PM
Subject: Re: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
On 1/13/2017 6:55 AM, Arjan van de Ven wrote:
> On 1/13/2017 6:41 AM, Marcos Felipe Schwarz wrote:
>> Further looking into the problem, the illegal instruction is shrx from bmi2 instruction set and this CPU don't support it.
>> So this CPU is not supported by Clear Linux by deafult?
>>
>
>
> so the dpdk makefiles sneak in an -march=native and our build servers are supporting BMI2
>
> will fix
>
fixed for the afternoon build today.
Hi Mario,
Thanks for the update. I could verify that it workend on a VM, but on a physical machine I keept getting the error: "Illegal instruction (core dumped)" when the openvswitch was running ovsdb-tool at the service initialization.
Using gdb I found out the problem was generated by DPDK:
"Program received signal SIGILL, Illegal instruction.
0x00007ffff6e350a8 in rte_cpu_get_flag_enabled () from /usr/lib64/librte_eal.so.2.1"
Recompiling the DPDK SRPM and reinstalling dpdk-lib made it work. The CPU from my equipment is an Atom C2758. Is this CPU not suposed to be supported by Clear Linux?
Regards,
Marcos Schwarz
----- Original Message -----
From: "Arevalo, Mario Alfredo C" <mario.alfredo.c.arevalo(a)intel.com>
To: "Marcos Felipe Schwarz" <marcos.schwarz(a)rnp.br>, "dev" <dev(a)lists.clearlinux.org>
Sent: Tuesday, January 10, 2017 9:28:38 PM
Subject: RE: DPDK support is openvswitch is broken and configuration documentation is outdated
Hi again Marcos Schwarz, now OVS-DPDK use case on Clear Linux should work, at this moment the new
configuration steps have not been updated in Clear Linux web site, they will come soon, however if you would
like to work with that as soon as possible, these are the new steps.
Regards.
Mario
***************** OVS-DPDK use case steps ****************************
1. Update ClearLinux release by a release greater than 12490 in host.
# swupd update
2. Install network-basic bundle
# swupd bundle-add network-basic
3. Enable VT-d technology in BIOS
4. Enable VT-d in host kernel command line, it is necessary to edit clear-linux-native-<current-
kernel-version>.conf file, and add iommu=pt intel_iommu=on to the end of the line. This file is in
UEFI boot partion.
# systemctl start boot.mount
# cd /boot/loader/entries/
5. Umount UEFI partition and reboot the machine.
6. Set number of hugepages.
# echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
7. Allocate pages on NUMA machines.
# echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
# echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
8. Make memory available for DPDK.
# mkdir -p /mnt/huge
# mount -t hugetlbfs nodev /mnt/huge
9. Download a clear linux image and OVMF.fd file, this image will be used as guest VMs
https://download.clearlinux.org/image/
10. Start OpenvSwitch service.
# systemctl start openvswitch
11. OpenvSwitch requires some configuration to enable DPDK functionality such as: core mask, socket
memory etc. this is and example in order to reproduce the environment in figure 1.0, more information
about DPDK configuration in http://docs.openvswitch.org/en/latest/
# ovs-vsctl --no-wait init
# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0x2
# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem=2048
# ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
12. Restart OpenvSwitch service in order to update new DPDK configuration.
# systemctl restart openvswitch13. Create a virtual bridge using openvswitch.
# ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
14. Add vhost dpdk ports to the bridge.
# ovs-vsctl add-port br0 vhost-user1 -- set Interface vhost-user1 type=dpdkvhostuser
# ovs-vsctl add-port br0 vhost-user2 -- set Interface vhost-user2 type=dpdkvhostuser
15. Run guest virtual machine A using the next configuration as reference, where $IMAGE var is the
name of the Clear Linux* OS for Intel Architecture image.
$ qemu-system-x86_64 \
-enable-kvm -m 1024 \
-bios OVMF.fd \
-smp 4 -cpu host \
-vga none -nographic \
-drive file="$IMAGE",if=virtio,aio=threads \
-chardev socket,id=char1,path=/run/openvswitch/vhost-user1 \
-netdev type=vhost-user,id=mynet1,chardev=char1,vhostforce \
-device virtio-net-pci,mac=00:00:00:00:00:01,netdev=mynet1 \
-object memory-backend-file,id=mem,size=1024M,mem-path=/dev/ hugepages,share=on \
-numa node,memdev=mem -mem-prealloc \
-debugcon file:debug.log -global isa-debugcon.iobase=0x402
16. Run guest virtual machine B, use the configuration from the previous step; simply change the MAC
address and the port socket. You can use 00:00:00:00:00:02 as a MAC address and vhost-user2 as a
socket.
17. Follow the instructions from https://clearlinux.org/documentation/ac-ovs-dpdk.html#setting-ip-
address
________________________________________
From: Dev [dev-bounces(a)lists.clearlinux.org] on behalf of Marcos Felipe Schwarz [marcos.schwarz(a)rnp.br]
Sent: Thursday, January 05, 2017 5:42 AM
To: dev(a)lists.clearlinux.org
Subject: [Clr-dev] DPDK support is openvswitch is broken and configuration documentation is outdated
Hi all.
I tried to use the openvswitch with dpdk in Clear Linux 12450, but noticed a few problems.
First, since version 2.6.0 the procedure to enable DPDK has changed and the flag --dpdk is not supported by the ovs-vswitchd command anymore, instead it is now necessary to configure DPDK using the ovs-vsctl command to set the other_config field with the desired parameters as detailed in https://github.com/openvswitch/ovs/blob/branch-2.6/INSTALL.DPDK.md. So the documentation in https://clearlinux.org/documentation/ac-ovs-dpdk.html should be updated. Exchanging the procedure number 7 from the "Using Linux OpenvSwitch-DPDK" section to the one detailed in the 2.6 documentation, ex:
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="2048,0"
Second, even with the correct commands, dpdk was not working, since the package is missing the flag "--with-dpdk" in the configure command of the SPEC file. After regenerating the package and using the correct procedure, the dpdk support in OVS worked as expected.
Regards,
Marcos Schwarz
Diretoria de Pesquisa & Desenvolvimento (DPD) / Research & Development Department
Gerência de Redes para Experimentos (GRE) / Management of Network for Experiments
RNP – Rede Nacional de Ensino e Pesquisa
http://www.rnp.br
Telefone: +55 (19) 3787 3386
E-mail: marcos.schwarz(a)rnp.br
_______________________________________________
Dev mailing list
Dev(a)lists.clearlinux.orghttps://lists.clearlinux.org/mailman/listinfo/dev
This client release enables mandatory signature verification, which means
the Manifest.MoM signature MUST verify correctly for an update operation to
continue, else swupd will exit. This ensures that wherever the content comes
from, clients are guaranteed to receive the "correct" content that they were
intended to consume. It also forces clients to remain on the secure update
path if only verifiable content is allowed on the system.
