mana: remove redundant doorbell in mana_poll_rx_cq()
With the last commit to refill the rx mbuf in batch, the doorbell
in mana_poll_rx_cq() becomes redundant. Remove it to save a few
microseconds spent in mmio call.
Reported by: NetApp
Reviewed by: Tallamraju, Sai
Tested by: whu
Fixes: 9b8701b8 ("mana: refill the rx mbuf in batch")
MFC after: 3 days
Sponsored by: Microsoft
mana: refill the rx mbuf in batch
Set the default refill threshod to be one quarter of the rx queue
length. User can change this value with hw.mana.rx_refill_thresh
in loader.conf. It improves the rx completion handling by saving
10% to 15% of overall time with this change.
Tested by: whu
MFC after: 2 weeks
Sponsored by: Microsoft
(cherry picked from commit 9b8701b81f14f0fa0787425eb9761b765d5faab0)
mana: Increase default tx and rx ring size to 1024
Tcp perfomance tests show high number of retries under heave tx
traffic. The numbers of queue stops and wakeups also increase.
Further analysis suggests the FreeBSD network stack tends to send
TSO packets with multiple sg entries, typically ranging from
10 to 16. On mana, every two sgs takes one unit of tx ring.
Therefore, adding up one unit for the head, it takes 6 to 9 units
of tx ring to send a typical TSO packet.
Current default tx ring size is 256, which can get filled up
quickly under heavy load. When tx ring is full, the send queue
is stopped waiting for the ring space to be freed. This could
cause the network stack to drop packets, and lead to tcp
retransmissions.
Increase the default tx and rx ring size to 1024 units. Also
introduce two tuneables allowing users to request tx and rx ring
size in loader.conf:
[14 lines not shown]mana: refill the rx mbuf in batch
Set the default refill threshod to be one quarter of the rx queue
length. User can change this value with hw.mana.rx_refill_thresh
in loader.conf. It improves the rx completion handling by saving
10% to 15% of overall time with this change.
Tested by: whu
MFC after: 2 weeks
Sponsored by: Microsoft
mana: Increase default tx and rx ring size to 1024
Tcp perfomance tests show high number of retries under heave tx
traffic. The numbers of queue stops and wakeups also increase.
Further analysis suggests the FreeBSD network stack tends to send
TSO packets with multiple sg entries, typically ranging from
10 to 16. On mana, every two sgs takes one unit of tx ring.
Therefore, adding up one unit for the head, it takes 6 to 9 units
of tx ring to send a typical TSO packet.
Current default tx ring size is 256, which can get filled up
quickly under heavy load. When tx ring is full, the send queue
is stopped waiting for the ring space to be freed. This could
cause the network stack to drop packets, and lead to tcp
retransmissions.
Increase the default tx and rx ring size to 1024 units. Also
introduce two tuneables allowing users to request tx and rx ring
size in loader.conf:
[12 lines not shown]mana: Remove stray semicolons
MFC after: 1 week
(cherry picked from commit 6ccf4f4071c5bf85a9aad593e92d1623e949c039)
mana: Remove stray semicolons
MFC after: 1 week
mana: Stop checking for failures from malloc/mallocarray/buf_ring_alloc(M_WAITOK)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45852
(cherry picked from commit 1dc7a7b74b5ad37ff7c8dc22f1a710460a5f1dcd)
mana: Stop checking for failures from malloc/mallocarray/buf_ring_alloc(M_WAITOK)
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D45852
net: Remove unneeded NULL check for the allocated ifnet
Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().
No functional change intended.
Reviewed by: kp, imp, glebius, stevek
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45740
(cherry picked from commit aa3860851b9f6a6002d135b1cac7736e0995eedc)
net: Remove unneeded NULL check for the allocated ifnet
Change 4787572d0580 made if_alloc_domain() never fail, then also do the
wrappers if_alloc(), if_alloc_dev(), and if_gethandle().
No functional change intended.
Reviewed by: kp, imp, glebius, stevek
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D45740
mana: Fix TX CQE error handling
For an unknown TX CQE error type (probably from a newer hardware),
still free the mbuf, update the queue tail, etc., otherwise the
accounting will be wrong.
Also, TX errors can be triggered by injecting corrupted packets, so
replace the mana_err to mana_dbg logging.
Reported by: NetApp
MFC after: 1 week
Sponsored by: Microsoft
(cherry picked from commit 516b5059705b6b8bbba28821dbe05964c128f9a9)
mana: Fix TX CQE error handling
For an unknown TX CQE error type (probably from a newer hardware),
still free the mbuf, update the queue tail, etc., otherwise the
accounting will be wrong.
Also, TX errors can be triggered by injecting corrupted packets, so
replace the mana_err to mana_dbg logging.
Reported by: NetApp
MFC after: 1 week
Sponsored by: Microsoft
FreeBSD/src fdafd31 — sys/dev/malo if_malo_pci.c if_malohal.c, sys/dev/mwl if_mwl_pci.c
sys: Automated cleanup of cdefs and other formatting
Apply the following automated changes to try to eliminate
no-longer-needed sys/cdefs.h includes as well as now-empty
blank lines in a row.
Remove /^#if.*\n#endif.*\n#include\s+<sys/cdefs.h>.*\n/
Remove /\n+#include\s+<sys/cdefs.h>.*\n+#if.*\n#endif.*\n+/
Remove /\n+#if.*\n#endif.*\n+/
Remove /^#if.*\n#endif.*\n/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/types.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/param.h>/
Remove /\n+#include\s+<sys/cdefs.h>\n#include\s+<sys/capsicum.h>/
Sponsored by: Netflix
mana: add lro and tso stat counters
Add a few stat counters for tso and lro.
Approved by: re (gjb)
Sponsored by: Microsoft
(cherry picked from commit b167e449c8db01f082691503fb5c1255ad5750eb)
(cherry picked from commit a72a0af8194effa5f7e7a88d09673ed798819e88)
mana: add ioctl to support toggling offloading features
With this support, users can enable or disable offloading features
such as txcsum, rxcsum, tso and software lro through ifconfig.
To enable or disable tx features, do it on mana interface first,
then hn/netvsc to sync it up with mana. For example:
ifconfig mana0 -txcsum
ifconfig hn0 -tscsum
To enable or disable rx features, just applying on mana interface
would be sufficient.
Disabling txcsum imples disabling tso. Enabling tso when txcsum
is disabled will result in an error message in dmesg requesting
to enable txcsum first.
Above applies to ipv6 offloading features as well.
[7 lines not shown]mana: add lro and tso stat counters
Add a few stat counters for tso and lro.
MFC after: 3 days
Sponsored by: Microsoft
(cherry picked from commit b167e449c8db01f082691503fb5c1255ad5750eb)
mana: add ioctl to support toggling offloading features
With this support, users can enable or disable offloading features
such as txcsum, rxcsum, tso and software lro through ifconfig.
To enable or disable tx features, do it on mana interface first,
then hn/netvsc to sync it up with mana. For example:
ifconfig mana0 -txcsum
ifconfig hn0 -tscsum
To enable or disable rx features, just applying on mana interface
would be sufficient.
Disabling txcsum imples disabling tso. Enabling tso when txcsum
is disabled will result in an error message in dmesg requesting
to enable txcsum first.
Above applies to ipv6 offloading features as well.
[6 lines not shown]mana: fix tso parameters and set hwassist bits
The parameters for tso on mana were not set correctly. Also the
hwassist bits were not set. These two cause tso on mana not work.
Fixed the issues and make tso working on mana.
Approved by: re (gjb)
Tested by: whu
Sponsored by: Microsoft
(cherry picked from commit 643fd7b4bc57de87ddfeb75a8f0bdb27dbb8c3ce)
(cherry picked from commit 4b22565f8598f2890d41e179fd36a0845f781262)
mana: add lro and tso stat counters
Add a few stat counters for tso and lro.
MFC after: 3 days
Sponsored by: Microsoft
mana: add ioctl to support toggling offloading features
With this support, users can enable or disable offloading features
such as txcsum, rxcsum, tso and software lro through ifconfig.
To enable or disable tx features, do it on mana interface first,
then hn/netvsc to sync it up with mana. For example:
ifconfig mana0 -txcsum
ifconfig hn0 -tscsum
To enable or disable rx features, just applying on mana interface
would be sufficient.
Disabling txcsum imples disabling tso. Enabling tso when txcsum
is disabled will result in an error message in dmesg requesting
to enable txcsum first.
Above applies to ipv6 offloading features as well.
[4 lines not shown]mana: fix tso parameters and set hwassist bits
The parameters for tso on mana were not set correctly. Also the
hwassist bits were not set. These two cause tso on mana not work.
Fixed the issues and make tso working on mana.
Tested by: whu
MFC after: 3 days
Sponsored by: Microsoft
(cherry picked from commit 643fd7b4bc57de87ddfeb75a8f0bdb27dbb8c3ce)
mana: batch ringing RX queue doorbell on receiving packets
It's inefficient to ring the doorbell page every time a WQE is posted to
the received queue. Excessive MMIO writes result in CPU spending more
time waiting on LOCK instructions (atomic operations), resulting in
poor scaling performance.
Move the code for ringing doorbell page to where after we have posted all
WQEs to the receive queue in mana_poll_rx_cq().
In addition, use the correct WQE count for ringing RQ doorbell.
The hardware specification specifies that WQE_COUNT should set to 0 for
the Receive Queue. Although currently the hardware doesn't enforce the
check, in the future releases it may check on this value.
Approved by: re (gjb)
Sponsored by: Microsoft
(cherry picked from commit e4e11c1d07f5d58ff8cf4e07ac8f61eecbbb5417)
(cherry picked from commit 55b7a8233e3da9355d555dff80011a97fac23079)
mana: batch ringing RX queue doorbell on receiving packets
It's inefficient to ring the doorbell page every time a WQE is posted to
the received queue. Excessive MMIO writes result in CPU spending more
time waiting on LOCK instructions (atomic operations), resulting in
poor scaling performance.
Move the code for ringing doorbell page to where after we have posted all
WQEs to the receive queue in mana_poll_rx_cq().
In addition, use the correct WQE count for ringing RQ doorbell.
The hardware specification specifies that WQE_COUNT should set to 0 for
the Receive Queue. Although currently the hardware doesn't enforce the
check, in the future releases it may check on this value.
Tested by: whu
MFC after: 1 week
Sponsored by: Microsoft
(cherry picked from commit e4e11c1d07f5d58ff8cf4e07ac8f61eecbbb5417)
mana: fix tso parameters and set hwassist bits
The parameters for tso on mana were not set correctly. Also the
hwassist bits were not set. These two cause tso on mana not work.
Fixed the issues and make tso working on mana.
Tested by: whu
MFC after: 3 days
Sponsored by: Microsoft
mana: batch ringing RX queue doorbell on receiving packets
It's inefficient to ring the doorbell page every time a WQE is posted to
the received queue. Excessive MMIO writes result in CPU spending more
time waiting on LOCK instructions (atomic operations), resulting in
poor scaling performance.
Move the code for ringing doorbell page to where after we have posted all
WQEs to the receive queue in mana_poll_rx_cq().
In addition, use the correct WQE count for ringing RQ doorbell.
The hardware specification specifies that WQE_COUNT should set to 0 for
the Receive Queue. Although currently the hardware doesn't enforce the
check, in the future releases it may check on this value.
Tested by: whu
MFC after: 1 week
Sponsored by: Microsoft
sys: Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
mana: fix a KASSERT panic on recursed lock access in mana_cfg_vport
The panic stack looks like this:
panic: _sx_xlock_hard: recursed on non-recursive sx MANA port lock
@ /usr/src/sys/dev/mana/mana_en.c:1022
KDB: stack backtrace:
vpanic() at vpanic+0x150/frame 0xfffffe011b3c1970
panic() at panic+0x43/frame 0xfffffe011b3c19d0
_sx_xlock_hard() at _sx_xlock_hard+0x82d/frame 0xfffffe011b3c1a70
_sx_xlock() at _sx_xlock+0xb0/frame 0xfffffe011b3c1ab0
mana_cfg_vport() at mana_cfg_vport+0x79/frame 0xfffffe011b3c1b40
mana_alloc_queues() at mana_alloc_queues+0x3b/frame 0xfffffe011b3c1c50
mana_up() at mana_up+0x40/frame 0xfffffe011b3c1c70
mana_ioctl() at mana_ioctl+0x25b/frame 0xfffffe011b3c1cb0
ifhwioctl() at ifhwioctl+0xd11/frame 0xfffffe011b3c1db0
hn_xpnt_vf_init() at hn_xpnt_vf_init+0x15f/frame 0xfffffe011b3c1e10
The lock has already been held in the caller. Remove this
[4 lines not shown]Mechanically convert mana to IfAPI
Reviewed by: zlei
Sponsored by: Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D37835
mana(4): Make the code cross-platform
Discussed with: whu
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D36388
