Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge branch 'phc-support-in-ena-driver'
David Arinzon says:
====================
PHC support in ENA driver
This patchset adds the support for PHC (PTP Hardware Clock)
in the ENA driver. The documentation part of the patchset
includes additional information, including statistics,
utilization and invocation examples through the testptp
utility.
====================
Link: https://patch.msgid.link/20250617110545.5659-1-darinzon@amazon.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
+1229
-10
+108
Documentation/networking/device_drivers/ethernet/amazon/ena.rst
+108
Documentation/networking/device_drivers/ethernet/amazon/ena.rst
···
56
56
ena_ethtool.c ethtool callbacks.
57
57
ena_xdp.[ch] XDP files
58
58
ena_pci_id_tbl.h Supported device IDs.
59
+
ena_phc.[ch] PTP hardware clock infrastructure (see `PHC`_ for more info)
60
+
ena_devlink.[ch] devlink files.
61
+
ena_debugfs.[ch] debugfs files.
59
62
================= ======================================================
60
63
61
64
Management Interface:
···
224
221
than RX copybreak bytes is received, it is copied into a new memory
225
222
buffer and the RX descriptor is returned to HW.
226
223
224
+
.. _`PHC`:
225
+
226
+
PTP Hardware Clock (PHC)
227
+
========================
228
+
.. _`ptp-userspace-api`: https://docs.kernel.org/driver-api/ptp.html#ptp-hardware-clock-user-space-api
229
+
.. _`testptp`: https://elixir.bootlin.com/linux/latest/source/tools/testing/selftests/ptp/testptp.c
230
+
231
+
ENA Linux driver supports PTP hardware clock providing timestamp reference to achieve nanosecond resolution.
232
+
233
+
**PHC support**
234
+
235
+
PHC depends on the PTP module, which needs to be either loaded as a module or compiled into the kernel.
236
+
237
+
Verify if the PTP module is present:
238
+
239
+
.. code-block:: shell
240
+
241
+
grep -w '^CONFIG_PTP_1588_CLOCK=[ym]' /boot/config-`uname -r`
242
+
243
+
- If no output is provided, the ENA driver cannot be loaded with PHC support.
244
+
245
+
**PHC activation**
246
+
247
+
The feature is turned off by default, in order to turn the feature on, the ENA driver
248
+
can be loaded in the following way:
249
+
250
+
- devlink:
251
+
252
+
.. code-block:: shell
253
+
254
+
sudo devlink dev param set pci/<domain:bus:slot.function> name enable_phc value true cmode driverinit
255
+
sudo devlink dev reload pci/<domain:bus:slot.function>
256
+
# for example:
257
+
sudo devlink dev param set pci/0000:00:06.0 name enable_phc value true cmode driverinit
258
+
sudo devlink dev reload pci/0000:00:06.0
259
+
260
+
All available PTP clock sources can be tracked here:
261
+
262
+
.. code-block:: shell
263
+
264
+
ls /sys/class/ptp
265
+
266
+
PHC support and capabilities can be verified using ethtool:
267
+
268
+
.. code-block:: shell
269
+
270
+
ethtool -T <interface>
271
+
272
+
**PHC timestamp**
273
+
274
+
To retrieve PHC timestamp, use `ptp-userspace-api`_, usage example using `testptp`_:
275
+
276
+
.. code-block:: shell
277
+
278
+
testptp -d /dev/ptp$(ethtool -T <interface> | awk '/PTP Hardware Clock:/ {print $NF}') -k 1
279
+
280
+
PHC get time requests should be within reasonable bounds,
281
+
avoid excessive utilization to ensure optimal performance and efficiency.
282
+
The ENA device restricts the frequency of PHC get time requests to a maximum
283
+
of 125 requests per second. If this limit is surpassed, the get time request
284
+
will fail, leading to an increment in the phc_err_ts statistic.
285
+
286
+
**PHC statistics**
287
+
288
+
PHC can be monitored using debugfs (if mounted):
289
+
290
+
.. code-block:: shell
291
+
292
+
sudo cat /sys/kernel/debug/<domain:bus:slot.function>/phc_stats
293
+
294
+
# for example:
295
+
sudo cat /sys/kernel/debug/0000:00:06.0/phc_stats
296
+
297
+
PHC errors must remain below 1% of all PHC requests to maintain the desired level of accuracy and reliability
298
+
299
+
================= ======================================================
300
+
**phc_cnt** | Number of successful retrieved timestamps (below expire timeout).
301
+
**phc_exp** | Number of expired retrieved timestamps (above expire timeout).
302
+
**phc_skp** | Number of skipped get time attempts (during block period).
303
+
**phc_err_dv** | Number of failed get time attempts due to device errors (entering into block state).
304
+
**phc_err_ts** | Number of failed get time attempts due to timestamp errors (entering into block state),
305
+
| This occurs if driver exceeded the request limit or device received an invalid timestamp.
306
+
================= ======================================================
307
+
308
+
PHC timeouts:
309
+
310
+
================= ======================================================
311
+
**expire** | Max time for a valid timestamp retrieval, passing this threshold will fail
312
+
| the get time request and block new requests until block timeout.
313
+
**block** | Blocking period starts once get time request expires or fails,
314
+
| all get time requests during block period will be skipped.
315
+
================= ======================================================
316
+
227
317
Statistics
228
318
==========
229
319
···
363
267
SKB.
364
268
- The user can provide a hash key, hash function, and configure the
365
269
indirection table through `ethtool(8)`.
270
+
271
+
DEVLINK SUPPORT
272
+
===============
273
+
.. _`devlink`: https://www.kernel.org/doc/html/latest/networking/devlink/index.html
274
+
275
+
`devlink`_ supports reloading the driver and initiating re-negotiation with the ENA device
276
+
277
+
.. code-block:: shell
278
+
279
+
sudo devlink dev reload pci/<domain:bus:slot.function>
280
+
# for example:
281
+
sudo devlink dev reload pci/0000:00:06.0
366
282
367
283
DATA PATH
368
284
=========
+3
Documentation/networking/devlink/devlink-params.rst
+3
Documentation/networking/devlink/devlink-params.rst
+2
drivers/net/ethernet/amazon/Kconfig
+2
drivers/net/ethernet/amazon/Kconfig
+1
-1
drivers/net/ethernet/amazon/ena/Makefile
+1
-1
drivers/net/ethernet/amazon/ena/Makefile
+73
-1
drivers/net/ethernet/amazon/ena/ena_admin_defs.h
+73
-1
drivers/net/ethernet/amazon/ena/ena_admin_defs.h
···
60
60
ENA_ADMIN_AENQ_CONFIG = 26,
61
61
ENA_ADMIN_LINK_CONFIG = 27,
62
62
ENA_ADMIN_HOST_ATTR_CONFIG = 28,
63
+
ENA_ADMIN_PHC_CONFIG = 29,
63
64
ENA_ADMIN_FEATURES_OPCODE_NUM = 32,
64
65
};
65
66
···
126
125
enum ena_admin_get_stats_scope {
127
126
ENA_ADMIN_SPECIFIC_QUEUE = 0,
128
127
ENA_ADMIN_ETH_TRAFFIC = 1,
128
+
};
129
+
130
+
enum ena_admin_phc_type {
131
+
ENA_ADMIN_PHC_TYPE_READLESS = 0,
132
+
};
133
+
134
+
enum ena_admin_phc_error_flags {
135
+
ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP = BIT(0),
129
136
};
130
137
131
138
/* ENA SRD configuration for ENI */
···
952
943
* 4 : rss_configurable_function_key
953
944
* 5 : reserved
954
945
* 6 : rx_page_reuse
955
-
* 31:7 : reserved
946
+
* 7 : reserved
947
+
* 8 : phc
948
+
* 31:9 : reserved
956
949
*/
957
950
u32 driver_supported_features;
958
951
};
···
1034
1023
};
1035
1024
};
1036
1025
1026
+
struct ena_admin_feature_phc_desc {
1027
+
/* PHC type as defined in enum ena_admin_get_phc_type,
1028
+
* used only for GET command.
1029
+
*/
1030
+
u8 type;
1031
+
1032
+
/* Reserved - MBZ */
1033
+
u8 reserved1[3];
1034
+
1035
+
/* PHC doorbell address as an offset to PCIe MMIO REG BAR,
1036
+
* used only for GET command.
1037
+
*/
1038
+
u32 doorbell_offset;
1039
+
1040
+
/* Max time for valid PHC retrieval, passing this threshold will
1041
+
* fail the get-time request and block PHC requests for
1042
+
* block_timeout_usec, used only for GET command.
1043
+
*/
1044
+
u32 expire_timeout_usec;
1045
+
1046
+
/* PHC requests block period, blocking starts if PHC request expired
1047
+
* in order to prevent floods on busy device,
1048
+
* used only for GET command.
1049
+
*/
1050
+
u32 block_timeout_usec;
1051
+
1052
+
/* Shared PHC physical address (ena_admin_phc_resp),
1053
+
* used only for SET command.
1054
+
*/
1055
+
struct ena_common_mem_addr output_address;
1056
+
1057
+
/* Shared PHC Size (ena_admin_phc_resp),
1058
+
* used only for SET command.
1059
+
*/
1060
+
u32 output_length;
1061
+
};
1062
+
1037
1063
struct ena_admin_get_feat_resp {
1038
1064
struct ena_admin_acq_common_desc acq_common_desc;
1039
1065
···
1100
1052
struct ena_admin_feature_intr_moder_desc intr_moderation;
1101
1053
1102
1054
struct ena_admin_ena_hw_hints hw_hints;
1055
+
1056
+
struct ena_admin_feature_phc_desc phc;
1103
1057
} u;
1104
1058
};
1105
1059
···
1135
1085
1136
1086
/* LLQ configuration */
1137
1087
struct ena_admin_feature_llq_desc llq;
1088
+
1089
+
/* PHC configuration */
1090
+
struct ena_admin_feature_phc_desc phc;
1138
1091
} u;
1139
1092
};
1140
1093
···
1213
1160
1214
1161
/* value is valid when poll is cleared */
1215
1162
u32 reg_val;
1163
+
};
1164
+
1165
+
struct ena_admin_phc_resp {
1166
+
/* Request Id, received from DB register */
1167
+
u16 req_id;
1168
+
1169
+
u8 reserved1[6];
1170
+
1171
+
/* PHC timestamp (nsec) */
1172
+
u64 timestamp;
1173
+
1174
+
u8 reserved2[12];
1175
+
1176
+
/* Bit field of enum ena_admin_phc_error_flags */
1177
+
u32 error_flags;
1178
+
1179
+
u8 reserved3[32];
1216
1180
};
1217
1181
1218
1182
/* aq_common_desc */
···
1330
1260
#define ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_MASK BIT(4)
1331
1261
#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_SHIFT 6
1332
1262
#define ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK BIT(6)
1263
+
#define ENA_ADMIN_HOST_INFO_PHC_SHIFT 8
1264
+
#define ENA_ADMIN_HOST_INFO_PHC_MASK BIT(8)
1333
1265
1334
1266
/* aenq_common_desc */
1335
1267
#define ENA_ADMIN_AENQ_COMMON_DESC_PHASE_MASK BIT(0)
+267
drivers/net/ethernet/amazon/ena/ena_com.c
+267
drivers/net/ethernet/amazon/ena/ena_com.c
···
41
41
42
42
#define ENA_MAX_ADMIN_POLL_US 5000
43
43
44
+
/* PHC definitions */
45
+
#define ENA_PHC_DEFAULT_EXPIRE_TIMEOUT_USEC 10
46
+
#define ENA_PHC_DEFAULT_BLOCK_TIMEOUT_USEC 1000
47
+
#define ENA_PHC_REQ_ID_OFFSET 0xDEAD
48
+
#define ENA_PHC_ERROR_FLAGS (ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP)
49
+
44
50
/*****************************************************************************/
45
51
/*****************************************************************************/
46
52
/*****************************************************************************/
···
1645
1639
1646
1640
writel(mask_value, ena_dev->reg_bar + ENA_REGS_INTR_MASK_OFF);
1647
1641
ena_dev->admin_queue.polling = polling;
1642
+
}
1643
+
1644
+
bool ena_com_phc_supported(struct ena_com_dev *ena_dev)
1645
+
{
1646
+
return ena_com_check_supported_feature_id(ena_dev, ENA_ADMIN_PHC_CONFIG);
1647
+
}
1648
+
1649
+
int ena_com_phc_init(struct ena_com_dev *ena_dev)
1650
+
{
1651
+
struct ena_com_phc_info *phc = &ena_dev->phc;
1652
+
1653
+
memset(phc, 0x0, sizeof(*phc));
1654
+
1655
+
/* Allocate shared mem used PHC timestamp retrieved from device */
1656
+
phc->virt_addr = dma_alloc_coherent(ena_dev->dmadev,
1657
+
sizeof(*phc->virt_addr),
1658
+
&phc->phys_addr,
1659
+
GFP_KERNEL);
1660
+
if (unlikely(!phc->virt_addr))
1661
+
return -ENOMEM;
1662
+
1663
+
spin_lock_init(&phc->lock);
1664
+
1665
+
phc->virt_addr->req_id = 0;
1666
+
phc->virt_addr->timestamp = 0;
1667
+
1668
+
return 0;
1669
+
}
1670
+
1671
+
int ena_com_phc_config(struct ena_com_dev *ena_dev)
1672
+
{
1673
+
struct ena_com_phc_info *phc = &ena_dev->phc;
1674
+
struct ena_admin_get_feat_resp get_feat_resp;
1675
+
struct ena_admin_set_feat_resp set_feat_resp;
1676
+
struct ena_admin_set_feat_cmd set_feat_cmd;
1677
+
int ret = 0;
1678
+
1679
+
/* Get device PHC default configuration */
1680
+
ret = ena_com_get_feature(ena_dev,
1681
+
&get_feat_resp,
1682
+
ENA_ADMIN_PHC_CONFIG,
1683
+
0);
1684
+
if (unlikely(ret)) {
1685
+
netdev_err(ena_dev->net_device,
1686
+
"Failed to get PHC feature configuration, error: %d\n",
1687
+
ret);
1688
+
return ret;
1689
+
}
1690
+
1691
+
/* Supporting only readless PHC retrieval */
1692
+
if (get_feat_resp.u.phc.type != ENA_ADMIN_PHC_TYPE_READLESS) {
1693
+
netdev_err(ena_dev->net_device,
1694
+
"Unsupported PHC type, error: %d\n",
1695
+
-EOPNOTSUPP);
1696
+
return -EOPNOTSUPP;
1697
+
}
1698
+
1699
+
/* Update PHC doorbell offset according to device value,
1700
+
* used to write req_id to PHC bar
1701
+
*/
1702
+
phc->doorbell_offset = get_feat_resp.u.phc.doorbell_offset;
1703
+
1704
+
/* Update PHC expire timeout according to device
1705
+
* or default driver value
1706
+
*/
1707
+
phc->expire_timeout_usec = (get_feat_resp.u.phc.expire_timeout_usec) ?
1708
+
get_feat_resp.u.phc.expire_timeout_usec :
1709
+
ENA_PHC_DEFAULT_EXPIRE_TIMEOUT_USEC;
1710
+
1711
+
/* Update PHC block timeout according to device
1712
+
* or default driver value
1713
+
*/
1714
+
phc->block_timeout_usec = (get_feat_resp.u.phc.block_timeout_usec) ?
1715
+
get_feat_resp.u.phc.block_timeout_usec :
1716
+
ENA_PHC_DEFAULT_BLOCK_TIMEOUT_USEC;
1717
+
1718
+
/* Sanity check - expire timeout must not exceed block timeout */
1719
+
if (phc->expire_timeout_usec > phc->block_timeout_usec)
1720
+
phc->expire_timeout_usec = phc->block_timeout_usec;
1721
+
1722
+
/* Prepare PHC feature command */
1723
+
memset(&set_feat_cmd, 0x0, sizeof(set_feat_cmd));
1724
+
set_feat_cmd.aq_common_descriptor.opcode = ENA_ADMIN_SET_FEATURE;
1725
+
set_feat_cmd.feat_common.feature_id = ENA_ADMIN_PHC_CONFIG;
1726
+
set_feat_cmd.u.phc.output_length = sizeof(*phc->virt_addr);
1727
+
ret = ena_com_mem_addr_set(ena_dev,
1728
+
&set_feat_cmd.u.phc.output_address,
1729
+
phc->phys_addr);
1730
+
if (unlikely(ret)) {
1731
+
netdev_err(ena_dev->net_device,
1732
+
"Failed setting PHC output address, error: %d\n",
1733
+
ret);
1734
+
return ret;
1735
+
}
1736
+
1737
+
/* Send PHC feature command to the device */
1738
+
ret = ena_com_execute_admin_command(&ena_dev->admin_queue,
1739
+
(struct ena_admin_aq_entry *)&set_feat_cmd,
1740
+
sizeof(set_feat_cmd),
1741
+
(struct ena_admin_acq_entry *)&set_feat_resp,
1742
+
sizeof(set_feat_resp));
1743
+
1744
+
if (unlikely(ret)) {
1745
+
netdev_err(ena_dev->net_device,
1746
+
"Failed to enable PHC, error: %d\n",
1747
+
ret);
1748
+
return ret;
1749
+
}
1750
+
1751
+
phc->active = true;
1752
+
netdev_dbg(ena_dev->net_device, "PHC is active in the device\n");
1753
+
1754
+
return ret;
1755
+
}
1756
+
1757
+
void ena_com_phc_destroy(struct ena_com_dev *ena_dev)
1758
+
{
1759
+
struct ena_com_phc_info *phc = &ena_dev->phc;
1760
+
unsigned long flags = 0;
1761
+
1762
+
/* In case PHC is not supported by the device, silently exiting */
1763
+
if (!phc->virt_addr)
1764
+
return;
1765
+
1766
+
spin_lock_irqsave(&phc->lock, flags);
1767
+
phc->active = false;
1768
+
spin_unlock_irqrestore(&phc->lock, flags);
1769
+
1770
+
dma_free_coherent(ena_dev->dmadev,
1771
+
sizeof(*phc->virt_addr),
1772
+
phc->virt_addr,
1773
+
phc->phys_addr);
1774
+
phc->virt_addr = NULL;
1775
+
}
1776
+
1777
+
int ena_com_phc_get_timestamp(struct ena_com_dev *ena_dev, u64 *timestamp)
1778
+
{
1779
+
volatile struct ena_admin_phc_resp *resp = ena_dev->phc.virt_addr;
1780
+
const ktime_t zero_system_time = ktime_set(0, 0);
1781
+
struct ena_com_phc_info *phc = &ena_dev->phc;
1782
+
ktime_t expire_time;
1783
+
ktime_t block_time;
1784
+
unsigned long flags = 0;
1785
+
int ret = 0;
1786
+
1787
+
if (!phc->active) {
1788
+
netdev_err(ena_dev->net_device, "PHC feature is not active in the device\n");
1789
+
return -EOPNOTSUPP;
1790
+
}
1791
+
1792
+
spin_lock_irqsave(&phc->lock, flags);
1793
+
1794
+
/* Check if PHC is in blocked state */
1795
+
if (unlikely(ktime_compare(phc->system_time, zero_system_time))) {
1796
+
/* Check if blocking time expired */
1797
+
block_time = ktime_add_us(phc->system_time, phc->block_timeout_usec);
1798
+
if (!ktime_after(ktime_get(), block_time)) {
1799
+
/* PHC is still in blocked state, skip PHC request */
1800
+
phc->stats.phc_skp++;
1801
+
ret = -EBUSY;
1802
+
goto skip;
1803
+
}
1804
+
1805
+
/* PHC is in active state, update statistics according
1806
+
* to req_id and error_flags
1807
+
*/
1808
+
if (READ_ONCE(resp->req_id) != phc->req_id) {
1809
+
/* Device didn't update req_id during blocking time,
1810
+
* this indicates on a device error
1811
+
*/
1812
+
netdev_err(ena_dev->net_device,
1813
+
"PHC get time request 0x%x failed (device error)\n",
1814
+
phc->req_id);
1815
+
phc->stats.phc_err_dv++;
1816
+
} else if (resp->error_flags & ENA_PHC_ERROR_FLAGS) {
1817
+
/* Device updated req_id during blocking time but got
1818
+
* a PHC error, this occurs if device:
1819
+
* - exceeded the get time request limit
1820
+
* - received an invalid timestamp
1821
+
*/
1822
+
netdev_err(ena_dev->net_device,
1823
+
"PHC get time request 0x%x failed (error 0x%x)\n",
1824
+
phc->req_id,
1825
+
resp->error_flags);
1826
+
phc->stats.phc_err_ts += !!(resp->error_flags &
1827
+
ENA_ADMIN_PHC_ERROR_FLAG_TIMESTAMP);
1828
+
} else {
1829
+
/* Device updated req_id during blocking time
1830
+
* with valid timestamp
1831
+
*/
1832
+
phc->stats.phc_exp++;
1833
+
}
1834
+
}
1835
+
1836
+
/* Setting relative timeouts */
1837
+
phc->system_time = ktime_get();
1838
+
block_time = ktime_add_us(phc->system_time, phc->block_timeout_usec);
1839
+
expire_time = ktime_add_us(phc->system_time, phc->expire_timeout_usec);
1840
+
1841
+
/* We expect the device to return this req_id once
1842
+
* the new PHC timestamp is updated
1843
+
*/
1844
+
phc->req_id++;
1845
+
1846
+
/* Initialize PHC shared memory with different req_id value
1847
+
* to be able to identify once the device changes it to req_id
1848
+
*/
1849
+
resp->req_id = phc->req_id + ENA_PHC_REQ_ID_OFFSET;
1850
+
1851
+
/* Writing req_id to PHC bar */
1852
+
writel(phc->req_id, ena_dev->reg_bar + phc->doorbell_offset);
1853
+
1854
+
/* Stalling until the device updates req_id */
1855
+
while (1) {
1856
+
if (unlikely(ktime_after(ktime_get(), expire_time))) {
1857
+
/* Gave up waiting for updated req_id, PHC enters into
1858
+
* blocked state until passing blocking time,
1859
+
* during this time any get PHC timestamp will fail with
1860
+
* device busy error
1861
+
*/
1862
+
ret = -EBUSY;
1863
+
break;
1864
+
}
1865
+
1866
+
/* Check if req_id was updated by the device */
1867
+
if (READ_ONCE(resp->req_id) != phc->req_id) {
1868
+
/* req_id was not updated by the device yet,
1869
+
* check again on next loop
1870
+
*/
1871
+
continue;
1872
+
}
1873
+
1874
+
/* req_id was updated by the device which indicates that
1875
+
* PHC timestamp and error_flags are updated too,
1876
+
* checking errors before retrieving timestamp
1877
+
*/
1878
+
if (unlikely(resp->error_flags & ENA_PHC_ERROR_FLAGS)) {
1879
+
/* Retrieved invalid PHC timestamp, PHC enters into
1880
+
* blocked state until passing blocking time,
1881
+
* during this time any get PHC timestamp requests
1882
+
* will fail with device busy error
1883
+
*/
1884
+
ret = -EBUSY;
1885
+
break;
1886
+
}
1887
+
1888
+
/* PHC timestamp value is returned to the caller */
1889
+
*timestamp = resp->timestamp;
1890
+
1891
+
/* Update statistic on valid PHC timestamp retrieval */
1892
+
phc->stats.phc_cnt++;
1893
+
1894
+
/* This indicates PHC state is active */
1895
+
phc->system_time = zero_system_time;
1896
+
break;
1897
+
}
1898
+
1899
+
skip:
1900
+
spin_unlock_irqrestore(&phc->lock, flags);
1901
+
1902
+
return ret;
1648
1903
}
1649
1904
1650
1905
int ena_com_mmio_reg_read_request_init(struct ena_com_dev *ena_dev)
+84
drivers/net/ethernet/amazon/ena/ena_com.h
+84
drivers/net/ethernet/amazon/ena/ena_com.h
···
210
210
u64 no_completion;
211
211
};
212
212
213
+
struct ena_com_stats_phc {
214
+
u64 phc_cnt;
215
+
u64 phc_exp;
216
+
u64 phc_skp;
217
+
u64 phc_err_dv;
218
+
u64 phc_err_ts;
219
+
};
220
+
213
221
struct ena_com_admin_queue {
214
222
void *q_dmadev;
215
223
struct ena_com_dev *ena_dev;
···
264
256
bool readless_supported;
265
257
/* spin lock to ensure a single outstanding read */
266
258
spinlock_t lock;
259
+
};
260
+
261
+
/* PTP hardware clock (PHC) MMIO read data info */
262
+
struct ena_com_phc_info {
263
+
/* Internal PHC statistics */
264
+
struct ena_com_stats_phc stats;
265
+
266
+
/* PHC shared memory - virtual address */
267
+
struct ena_admin_phc_resp *virt_addr;
268
+
269
+
/* System time of last PHC request */
270
+
ktime_t system_time;
271
+
272
+
/* Spin lock to ensure a single outstanding PHC read */
273
+
spinlock_t lock;
274
+
275
+
/* PHC doorbell address as an offset to PCIe MMIO REG BAR */
276
+
u32 doorbell_offset;
277
+
278
+
/* Shared memory read expire timeout (usec)
279
+
* Max time for valid PHC retrieval, passing this threshold will fail
280
+
* the get time request and block new PHC requests for block_timeout_usec
281
+
* in order to prevent floods on busy device
282
+
*/
283
+
u32 expire_timeout_usec;
284
+
285
+
/* Shared memory read abort timeout (usec)
286
+
* PHC requests block period, blocking starts once PHC request expired
287
+
* in order to prevent floods on busy device,
288
+
* any PHC requests during block period will be skipped
289
+
*/
290
+
u32 block_timeout_usec;
291
+
292
+
/* PHC shared memory - physical address */
293
+
dma_addr_t phys_addr;
294
+
295
+
/* Request id sent to the device */
296
+
u16 req_id;
297
+
298
+
/* True if PHC is active in the device */
299
+
bool active;
267
300
};
268
301
269
302
struct ena_rss {
···
366
317
u32 ena_min_poll_delay_us;
367
318
368
319
struct ena_com_mmio_read mmio_read;
320
+
struct ena_com_phc_info phc;
369
321
370
322
struct ena_rss rss;
371
323
u32 supported_features;
···
431
381
* @return - 0 on success, negative value on failure.
432
382
*/
433
383
int ena_com_mmio_reg_read_request_init(struct ena_com_dev *ena_dev);
384
+
385
+
/* ena_com_phc_init - Allocate and initialize PHC feature
386
+
* @ena_dev: ENA communication layer struct
387
+
* @note: This method assumes PHC is supported by the device
388
+
* @return - 0 on success, negative value on failure
389
+
*/
390
+
int ena_com_phc_init(struct ena_com_dev *ena_dev);
391
+
392
+
/* ena_com_phc_supported - Return if PHC feature is supported by the device
393
+
* @ena_dev: ENA communication layer struct
394
+
* @note: This method must be called after getting supported features
395
+
* @return - supported or not
396
+
*/
397
+
bool ena_com_phc_supported(struct ena_com_dev *ena_dev);
398
+
399
+
/* ena_com_phc_config - Configure PHC feature
400
+
* @ena_dev: ENA communication layer struct
401
+
* Configure PHC feature in driver and device
402
+
* @note: This method assumes PHC is supported by the device
403
+
* @return - 0 on success, negative value on failure
404
+
*/
405
+
int ena_com_phc_config(struct ena_com_dev *ena_dev);
406
+
407
+
/* ena_com_phc_destroy - Destroy PHC feature
408
+
* @ena_dev: ENA communication layer struct
409
+
*/
410
+
void ena_com_phc_destroy(struct ena_com_dev *ena_dev);
411
+
412
+
/* ena_com_phc_get_timestamp - Retrieve PHC timestamp
413
+
* @ena_dev: ENA communication layer struct
414
+
* @timestamp: Retrieved PHC timestamp
415
+
* @return - 0 on success, negative value on failure
416
+
*/
417
+
int ena_com_phc_get_timestamp(struct ena_com_dev *ena_dev, u64 *timestamp);
434
418
435
419
/* ena_com_set_mmio_read_mode - Enable/disable the indirect mmio reg read mechanism
436
420
* @ena_dev: ENA communication layer struct
+62
drivers/net/ethernet/amazon/ena/ena_debugfs.c
+62
drivers/net/ethernet/amazon/ena/ena_debugfs.c
···
1
+
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
2
+
/* Copyright (c) Amazon.com, Inc. or its affiliates.
3
+
* All rights reserved.
4
+
*/
5
+
6
+
#ifdef CONFIG_DEBUG_FS
7
+
8
+
#include <linux/seq_file.h>
9
+
#include <linux/pci.h>
10
+
#include "ena_debugfs.h"
11
+
#include "ena_phc.h"
12
+
13
+
static int phc_stats_show(struct seq_file *file, void *priv)
14
+
{
15
+
struct ena_adapter *adapter = file->private;
16
+
17
+
if (!ena_phc_is_active(adapter))
18
+
return 0;
19
+
20
+
seq_printf(file,
21
+
"phc_cnt: %llu\n",
22
+
adapter->ena_dev->phc.stats.phc_cnt);
23
+
seq_printf(file,
24
+
"phc_exp: %llu\n",
25
+
adapter->ena_dev->phc.stats.phc_exp);
26
+
seq_printf(file,
27
+
"phc_skp: %llu\n",
28
+
adapter->ena_dev->phc.stats.phc_skp);
29
+
seq_printf(file,
30
+
"phc_err_dv: %llu\n",
31
+
adapter->ena_dev->phc.stats.phc_err_dv);
32
+
seq_printf(file,
33
+
"phc_err_ts: %llu\n",
34
+
adapter->ena_dev->phc.stats.phc_err_ts);
35
+
36
+
return 0;
37
+
}
38
+
39
+
DEFINE_SHOW_ATTRIBUTE(phc_stats);
40
+
41
+
void ena_debugfs_init(struct net_device *dev)
42
+
{
43
+
struct ena_adapter *adapter = netdev_priv(dev);
44
+
45
+
adapter->debugfs_base =
46
+
debugfs_create_dir(dev_name(&adapter->pdev->dev), NULL);
47
+
48
+
debugfs_create_file("phc_stats",
49
+
0400,
50
+
adapter->debugfs_base,
51
+
adapter,
52
+
&phc_stats_fops);
53
+
}
54
+
55
+
void ena_debugfs_terminate(struct net_device *dev)
56
+
{
57
+
struct ena_adapter *adapter = netdev_priv(dev);
58
+
59
+
debugfs_remove_recursive(adapter->debugfs_base);
60
+
}
61
+
62
+
#endif /* CONFIG_DEBUG_FS */
+27
drivers/net/ethernet/amazon/ena/ena_debugfs.h
+27
drivers/net/ethernet/amazon/ena/ena_debugfs.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
2
+
/* Copyright (c) Amazon.com, Inc. or its affiliates.
3
+
* All rights reserved.
4
+
*/
5
+
6
+
#ifndef __ENA_DEBUGFS_H__
7
+
#define __ENA_DEBUGFS_H__
8
+
9
+
#include <linux/debugfs.h>
10
+
#include <linux/netdevice.h>
11
+
#include "ena_netdev.h"
12
+
13
+
#ifdef CONFIG_DEBUG_FS
14
+
15
+
void ena_debugfs_init(struct net_device *dev);
16
+
17
+
void ena_debugfs_terminate(struct net_device *dev);
18
+
19
+
#else /* CONFIG_DEBUG_FS */
20
+
21
+
static inline void ena_debugfs_init(struct net_device *dev) {}
22
+
23
+
static inline void ena_debugfs_terminate(struct net_device *dev) {}
24
+
25
+
#endif /* CONFIG_DEBUG_FS */
26
+
27
+
#endif /* __ENA_DEBUGFS_H__ */
+210
drivers/net/ethernet/amazon/ena/ena_devlink.c
+210
drivers/net/ethernet/amazon/ena/ena_devlink.c
···
1
+
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
2
+
/* Copyright (c) Amazon.com, Inc. or its affiliates.
3
+
* All rights reserved.
4
+
*/
5
+
6
+
#include "linux/pci.h"
7
+
#include "ena_devlink.h"
8
+
#include "ena_phc.h"
9
+
10
+
static int ena_devlink_enable_phc_validate(struct devlink *devlink, u32 id,
11
+
union devlink_param_value val,
12
+
struct netlink_ext_ack *extack)
13
+
{
14
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
15
+
16
+
if (!val.vbool)
17
+
return 0;
18
+
19
+
if (!ena_com_phc_supported(adapter->ena_dev)) {
20
+
NL_SET_ERR_MSG_MOD(extack, "Device doesn't support PHC");
21
+
return -EOPNOTSUPP;
22
+
}
23
+
24
+
return 0;
25
+
}
26
+
27
+
static const struct devlink_param ena_devlink_params[] = {
28
+
DEVLINK_PARAM_GENERIC(ENABLE_PHC,
29
+
BIT(DEVLINK_PARAM_CMODE_DRIVERINIT),
30
+
NULL,
31
+
NULL,
32
+
ena_devlink_enable_phc_validate),
33
+
};
34
+
35
+
void ena_devlink_params_get(struct devlink *devlink)
36
+
{
37
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
38
+
union devlink_param_value val;
39
+
int err;
40
+
41
+
err = devl_param_driverinit_value_get(devlink,
42
+
DEVLINK_PARAM_GENERIC_ID_ENABLE_PHC,
43
+
&val);
44
+
if (err) {
45
+
netdev_err(adapter->netdev, "Failed to query PHC param\n");
46
+
return;
47
+
}
48
+
49
+
ena_phc_enable(adapter, val.vbool);
50
+
}
51
+
52
+
void ena_devlink_disable_phc_param(struct devlink *devlink)
53
+
{
54
+
union devlink_param_value value;
55
+
56
+
value.vbool = false;
57
+
devl_param_driverinit_value_set(devlink,
58
+
DEVLINK_PARAM_GENERIC_ID_ENABLE_PHC,
59
+
value);
60
+
}
61
+
62
+
static void ena_devlink_port_register(struct devlink *devlink)
63
+
{
64
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
65
+
struct devlink_port_attrs attrs = {};
66
+
67
+
attrs.flavour = DEVLINK_PORT_FLAVOUR_PHYSICAL;
68
+
devlink_port_attrs_set(&adapter->devlink_port, &attrs);
69
+
devl_port_register(devlink, &adapter->devlink_port, 0);
70
+
}
71
+
72
+
static void ena_devlink_port_unregister(struct devlink *devlink)
73
+
{
74
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
75
+
76
+
devl_port_unregister(&adapter->devlink_port);
77
+
}
78
+
79
+
static int ena_devlink_reload_down(struct devlink *devlink,
80
+
bool netns_change,
81
+
enum devlink_reload_action action,
82
+
enum devlink_reload_limit limit,
83
+
struct netlink_ext_ack *extack)
84
+
{
85
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
86
+
87
+
if (netns_change) {
88
+
NL_SET_ERR_MSG_MOD(extack,
89
+
"Namespace change is not supported");
90
+
return -EOPNOTSUPP;
91
+
}
92
+
93
+
ena_devlink_port_unregister(devlink);
94
+
95
+
rtnl_lock();
96
+
ena_destroy_device(adapter, false);
97
+
rtnl_unlock();
98
+
99
+
return 0;
100
+
}
101
+
102
+
static int ena_devlink_reload_up(struct devlink *devlink,
103
+
enum devlink_reload_action action,
104
+
enum devlink_reload_limit limit,
105
+
u32 *actions_performed,
106
+
struct netlink_ext_ack *extack)
107
+
{
108
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
109
+
int err = 0;
110
+
111
+
rtnl_lock();
112
+
/* Check that no other routine initialized the device (e.g.
113
+
* ena_fw_reset_device()). Also we're under devlink_mutex here,
114
+
* so devlink isn't freed under our feet.
115
+
*/
116
+
if (!test_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags))
117
+
err = ena_restore_device(adapter);
118
+
119
+
rtnl_unlock();
120
+
121
+
ena_devlink_port_register(devlink);
122
+
123
+
if (!err)
124
+
*actions_performed = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT);
125
+
126
+
return err;
127
+
}
128
+
129
+
static const struct devlink_ops ena_devlink_ops = {
130
+
.reload_actions = BIT(DEVLINK_RELOAD_ACTION_DRIVER_REINIT),
131
+
.reload_down = ena_devlink_reload_down,
132
+
.reload_up = ena_devlink_reload_up,
133
+
};
134
+
135
+
static int ena_devlink_configure_params(struct devlink *devlink)
136
+
{
137
+
struct ena_adapter *adapter = ENA_DEVLINK_PRIV(devlink);
138
+
union devlink_param_value value;
139
+
int rc;
140
+
141
+
rc = devlink_params_register(devlink, ena_devlink_params,
142
+
ARRAY_SIZE(ena_devlink_params));
143
+
if (rc) {
144
+
netdev_err(adapter->netdev, "Failed to register devlink params\n");
145
+
return rc;
146
+
}
147
+
148
+
value.vbool = ena_phc_is_enabled(adapter);
149
+
devl_param_driverinit_value_set(devlink,
150
+
DEVLINK_PARAM_GENERIC_ID_ENABLE_PHC,
151
+
value);
152
+
153
+
return 0;
154
+
}
155
+
156
+
struct devlink *ena_devlink_alloc(struct ena_adapter *adapter)
157
+
{
158
+
struct device *dev = &adapter->pdev->dev;
159
+
struct devlink *devlink;
160
+
161
+
devlink = devlink_alloc(&ena_devlink_ops,
162
+
sizeof(struct ena_adapter *),
163
+
dev);
164
+
if (!devlink) {
165
+
netdev_err(adapter->netdev,
166
+
"Failed to allocate devlink struct\n");
167
+
return NULL;
168
+
}
169
+
170
+
ENA_DEVLINK_PRIV(devlink) = adapter;
171
+
adapter->devlink = devlink;
172
+
173
+
if (ena_devlink_configure_params(devlink))
174
+
goto free_devlink;
175
+
176
+
return devlink;
177
+
178
+
free_devlink:
179
+
devlink_free(devlink);
180
+
return NULL;
181
+
}
182
+
183
+
static void ena_devlink_configure_params_clean(struct devlink *devlink)
184
+
{
185
+
devlink_params_unregister(devlink, ena_devlink_params,
186
+
ARRAY_SIZE(ena_devlink_params));
187
+
}
188
+
189
+
void ena_devlink_free(struct devlink *devlink)
190
+
{
191
+
ena_devlink_configure_params_clean(devlink);
192
+
193
+
devlink_free(devlink);
194
+
}
195
+
196
+
void ena_devlink_register(struct devlink *devlink, struct device *dev)
197
+
{
198
+
devl_lock(devlink);
199
+
ena_devlink_port_register(devlink);
200
+
devl_register(devlink);
201
+
devl_unlock(devlink);
202
+
}
203
+
204
+
void ena_devlink_unregister(struct devlink *devlink)
205
+
{
206
+
devl_lock(devlink);
207
+
ena_devlink_port_unregister(devlink);
208
+
devl_unregister(devlink);
209
+
devl_unlock(devlink);
210
+
}
+21
drivers/net/ethernet/amazon/ena/ena_devlink.h
+21
drivers/net/ethernet/amazon/ena/ena_devlink.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
2
+
/* Copyright (c) Amazon.com, Inc. or its affiliates.
3
+
* All rights reserved.
4
+
*/
5
+
#ifndef DEVLINK_H
6
+
#define DEVLINK_H
7
+
8
+
#include "ena_netdev.h"
9
+
#include <net/devlink.h>
10
+
11
+
#define ENA_DEVLINK_PRIV(devlink) \
12
+
(*(struct ena_adapter **)devlink_priv(devlink))
13
+
14
+
struct devlink *ena_devlink_alloc(struct ena_adapter *adapter);
15
+
void ena_devlink_free(struct devlink *devlink);
16
+
void ena_devlink_register(struct devlink *devlink, struct device *dev);
17
+
void ena_devlink_unregister(struct devlink *devlink);
18
+
void ena_devlink_params_get(struct devlink *devlink);
19
+
void ena_devlink_disable_phc_param(struct devlink *devlink);
20
+
21
+
#endif /* DEVLINK_H */
+15
-1
drivers/net/ethernet/amazon/ena/ena_ethtool.c
+15
-1
drivers/net/ethernet/amazon/ena/ena_ethtool.c
···
5
5
6
6
#include <linux/ethtool.h>
7
7
#include <linux/pci.h>
8
+
#include <linux/net_tstamp.h>
8
9
9
10
#include "ena_netdev.h"
10
11
#include "ena_xdp.h"
12
+
#include "ena_phc.h"
11
13
12
14
struct ena_stats {
13
15
char name[ETH_GSTRING_LEN];
···
298
296
struct ena_adapter *adapter = netdev_priv(netdev);
299
297
300
298
ena_get_stats(adapter, data, true);
299
+
}
300
+
301
+
static int ena_get_ts_info(struct net_device *netdev,
302
+
struct kernel_ethtool_ts_info *info)
303
+
{
304
+
struct ena_adapter *adapter = netdev_priv(netdev);
305
+
306
+
info->so_timestamping = SOF_TIMESTAMPING_TX_SOFTWARE;
307
+
308
+
info->phc_index = ena_phc_get_index(adapter);
309
+
310
+
return 0;
301
311
}
302
312
303
313
static int ena_get_sw_stats_count(struct ena_adapter *adapter)
···
1104
1090
.set_channels = ena_set_channels,
1105
1091
.get_tunable = ena_get_tunable,
1106
1092
.set_tunable = ena_set_tunable,
1107
-
.get_ts_info = ethtool_op_get_ts_info,
1093
+
.get_ts_info = ena_get_ts_info,
1108
1094
};
1109
1095
1110
1096
void ena_set_ethtool_ops(struct net_device *netdev)
+55
-7
drivers/net/ethernet/amazon/ena/ena_netdev.c
+55
-7
drivers/net/ethernet/amazon/ena/ena_netdev.c
···
19
19
#include "ena_pci_id_tbl.h"
20
20
#include "ena_xdp.h"
21
21
22
+
#include "ena_phc.h"
23
+
24
+
#include "ena_devlink.h"
25
+
26
+
#include "ena_debugfs.h"
27
+
22
28
MODULE_AUTHOR("Amazon.com, Inc. or its affiliates");
23
29
MODULE_DESCRIPTION(DEVICE_NAME);
24
30
MODULE_LICENSE("GPL");
···
45
39
46
40
static int ena_rss_init_default(struct ena_adapter *adapter);
47
41
static void check_for_admin_com_state(struct ena_adapter *adapter);
48
-
static int ena_destroy_device(struct ena_adapter *adapter, bool graceful);
49
-
static int ena_restore_device(struct ena_adapter *adapter);
50
42
51
43
static void ena_tx_timeout(struct net_device *dev, unsigned int txqueue)
52
44
{
···
2747
2743
ENA_ADMIN_HOST_INFO_INTERRUPT_MODERATION_MASK |
2748
2744
ENA_ADMIN_HOST_INFO_RX_BUF_MIRRORING_MASK |
2749
2745
ENA_ADMIN_HOST_INFO_RSS_CONFIGURABLE_FUNCTION_KEY_MASK |
2750
-
ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK;
2746
+
ENA_ADMIN_HOST_INFO_RX_PAGE_REUSE_MASK |
2747
+
ENA_ADMIN_HOST_INFO_PHC_MASK;
2751
2748
2752
2749
rc = ena_com_set_host_attributes(ena_dev);
2753
2750
if (rc) {
···
3140
3135
goto err_mmio_read_less;
3141
3136
}
3142
3137
3138
+
ena_devlink_params_get(adapter->devlink);
3139
+
3143
3140
/* ENA admin level init */
3144
3141
rc = ena_com_admin_init(ena_dev, &aenq_handlers);
3145
3142
if (rc) {
···
3195
3188
if (unlikely(rc))
3196
3189
goto err_admin_init;
3197
3190
3191
+
rc = ena_phc_init(adapter);
3192
+
if (unlikely(rc && (rc != -EOPNOTSUPP)))
3193
+
netdev_err(netdev, "Failed initializing PHC, error: %d\n", rc);
3194
+
3198
3195
return 0;
3199
3196
3200
3197
err_admin_init:
···
3244
3233
return rc;
3245
3234
}
3246
3235
3247
-
static int ena_destroy_device(struct ena_adapter *adapter, bool graceful)
3236
+
int ena_destroy_device(struct ena_adapter *adapter, bool graceful)
3248
3237
{
3249
3238
struct net_device *netdev = adapter->netdev;
3250
3239
struct ena_com_dev *ena_dev = adapter->ena_dev;
···
3282
3271
3283
3272
ena_com_admin_destroy(ena_dev);
3284
3273
3274
+
ena_phc_destroy(adapter);
3275
+
3285
3276
ena_com_mmio_reg_read_request_destroy(ena_dev);
3286
3277
3287
3278
/* return reset reason to default value */
···
3295
3282
return rc;
3296
3283
}
3297
3284
3298
-
static int ena_restore_device(struct ena_adapter *adapter)
3285
+
int ena_restore_device(struct ena_adapter *adapter)
3299
3286
{
3300
3287
struct ena_com_dev_get_features_ctx get_feat_ctx;
3301
3288
struct ena_com_dev *ena_dev = adapter->ena_dev;
···
3357
3344
ena_com_wait_for_abort_completion(ena_dev);
3358
3345
ena_com_admin_destroy(ena_dev);
3359
3346
ena_com_dev_reset(ena_dev, ENA_REGS_RESET_DRIVER_INVALID_STATE);
3347
+
ena_phc_destroy(adapter);
3360
3348
ena_com_mmio_reg_read_request_destroy(ena_dev);
3361
3349
err:
3362
3350
clear_bit(ENA_FLAG_DEVICE_RUNNING, &adapter->flags);
···
3881
3867
struct ena_adapter *adapter;
3882
3868
struct net_device *netdev;
3883
3869
static int adapters_found;
3870
+
struct devlink *devlink;
3884
3871
u32 max_num_io_queues;
3885
3872
bool wd_state;
3886
3873
int bars, rc;
···
3947
3932
3948
3933
pci_set_drvdata(pdev, adapter);
3949
3934
3935
+
rc = ena_phc_alloc(adapter);
3936
+
if (rc) {
3937
+
netdev_err(netdev, "ena_phc_alloc failed\n");
3938
+
goto err_netdev_destroy;
3939
+
}
3940
+
3950
3941
rc = ena_com_allocate_customer_metrics_buffer(ena_dev);
3951
3942
if (rc) {
3952
3943
netdev_err(netdev, "ena_com_allocate_customer_metrics_buffer failed\n");
3953
-
goto err_netdev_destroy;
3944
+
goto err_free_phc;
3954
3945
}
3955
3946
3956
3947
rc = ena_map_llq_mem_bar(pdev, ena_dev, bars);
···
3965
3944
goto err_metrics_destroy;
3966
3945
}
3967
3946
3947
+
/* Need to do this before ena_device_init */
3948
+
devlink = ena_devlink_alloc(adapter);
3949
+
if (!devlink) {
3950
+
netdev_err(netdev, "ena_devlink_alloc failed\n");
3951
+
rc = -ENOMEM;
3952
+
goto err_metrics_destroy;
3953
+
}
3954
+
3968
3955
rc = ena_device_init(adapter, pdev, &get_feat_ctx, &wd_state);
3969
3956
if (rc) {
3970
3957
dev_err(&pdev->dev, "ENA device init failed\n");
3971
3958
if (rc == -ETIME)
3972
3959
rc = -EPROBE_DEFER;
3973
-
goto err_metrics_destroy;
3960
+
goto ena_devlink_destroy;
3974
3961
}
3975
3962
3976
3963
/* Initial TX and RX interrupt delay. Assumes 1 usec granularity.
···
4062
4033
goto err_rss;
4063
4034
}
4064
4035
4036
+
ena_debugfs_init(netdev);
4037
+
4065
4038
INIT_WORK(&adapter->reset_task, ena_fw_reset_device);
4066
4039
4067
4040
adapter->last_keep_alive_jiffies = jiffies;
···
4085
4054
4086
4055
adapters_found++;
4087
4056
4057
+
/* From this point, the devlink device is visible to users.
4058
+
* Perform the registration last to ensure that all the resources
4059
+
* are available and that the netdevice is registered.
4060
+
*/
4061
+
ena_devlink_register(devlink, &pdev->dev);
4062
+
4088
4063
return 0;
4089
4064
4090
4065
err_rss:
···
4107
4070
err_device_destroy:
4108
4071
ena_com_delete_host_info(ena_dev);
4109
4072
ena_com_admin_destroy(ena_dev);
4073
+
ena_devlink_destroy:
4074
+
ena_devlink_free(devlink);
4110
4075
err_metrics_destroy:
4111
4076
ena_com_delete_customer_metrics_buffer(ena_dev);
4077
+
err_free_phc:
4078
+
ena_phc_free(adapter);
4112
4079
err_netdev_destroy:
4113
4080
free_netdev(netdev);
4114
4081
err_free_region:
···
4143
4102
ena_dev = adapter->ena_dev;
4144
4103
netdev = adapter->netdev;
4145
4104
4105
+
ena_debugfs_terminate(netdev);
4106
+
4146
4107
/* Make sure timer and reset routine won't be called after
4147
4108
* freeing device resources.
4148
4109
*/
···
4154
4111
rtnl_lock(); /* lock released inside the below if-else block */
4155
4112
adapter->reset_reason = ENA_REGS_RESET_SHUTDOWN;
4156
4113
ena_destroy_device(adapter, true);
4114
+
4115
+
ena_phc_free(adapter);
4116
+
4117
+
ena_devlink_unregister(adapter->devlink);
4118
+
ena_devlink_free(adapter->devlink);
4157
4119
4158
4120
if (shutdown) {
4159
4121
netif_device_detach(netdev);
+14
drivers/net/ethernet/amazon/ena/ena_netdev.h
+14
drivers/net/ethernet/amazon/ena/ena_netdev.h
···
16
16
#include <linux/skbuff.h>
17
17
#include <net/xdp.h>
18
18
#include <uapi/linux/bpf.h>
19
+
#include <net/devlink.h>
19
20
20
21
#include "ena_com.h"
21
22
#include "ena_eth_com.h"
···
110
109
#define ENA_MAX_NO_INTERRUPT_ITERATIONS 3
111
110
112
111
#define ENA_MMIO_DISABLE_REG_READ BIT(0)
112
+
113
+
struct ena_phc_info;
113
114
114
115
struct ena_irq {
115
116
irq_handler_t handler;
···
351
348
352
349
char name[ENA_NAME_MAX_LEN];
353
350
351
+
struct ena_phc_info *phc_info;
352
+
354
353
unsigned long flags;
355
354
/* TX */
356
355
struct ena_ring tx_ring[ENA_MAX_NUM_IO_QUEUES]
···
388
383
struct bpf_prog *xdp_bpf_prog;
389
384
u32 xdp_first_ring;
390
385
u32 xdp_num_queues;
386
+
387
+
struct devlink *devlink;
388
+
struct devlink_port devlink_port;
389
+
#ifdef CONFIG_DEBUG_FS
390
+
391
+
struct dentry *debugfs_base;
392
+
#endif /* CONFIG_DEBUG_FS */
391
393
};
392
394
393
395
void ena_set_ethtool_ops(struct net_device *netdev);
···
424
412
set_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags);
425
413
}
426
414
415
+
int ena_destroy_device(struct ena_adapter *adapter, bool graceful);
416
+
int ena_restore_device(struct ena_adapter *adapter);
427
417
int handle_invalid_req_id(struct ena_ring *ring, u16 req_id,
428
418
struct ena_tx_buffer *tx_info, bool is_xdp);
429
419
+233
drivers/net/ethernet/amazon/ena/ena_phc.c
+233
drivers/net/ethernet/amazon/ena/ena_phc.c
···
1
+
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
2
+
/*
3
+
* Copyright 2015-2022 Amazon.com, Inc. or its affiliates. All rights reserved.
4
+
*/
5
+
6
+
#include <linux/pci.h>
7
+
#include "ena_netdev.h"
8
+
#include "ena_phc.h"
9
+
#include "ena_devlink.h"
10
+
11
+
static int ena_phc_adjtime(struct ptp_clock_info *clock_info, s64 delta)
12
+
{
13
+
return -EOPNOTSUPP;
14
+
}
15
+
16
+
static int ena_phc_adjfine(struct ptp_clock_info *clock_info, long scaled_ppm)
17
+
{
18
+
return -EOPNOTSUPP;
19
+
}
20
+
21
+
static int ena_phc_feature_enable(struct ptp_clock_info *clock_info,
22
+
struct ptp_clock_request *rq,
23
+
int on)
24
+
{
25
+
return -EOPNOTSUPP;
26
+
}
27
+
28
+
static int ena_phc_gettimex64(struct ptp_clock_info *clock_info,
29
+
struct timespec64 *ts,
30
+
struct ptp_system_timestamp *sts)
31
+
{
32
+
struct ena_phc_info *phc_info =
33
+
container_of(clock_info, struct ena_phc_info, clock_info);
34
+
unsigned long flags;
35
+
u64 timestamp_nsec;
36
+
int rc;
37
+
38
+
spin_lock_irqsave(&phc_info->lock, flags);
39
+
40
+
ptp_read_system_prets(sts);
41
+
42
+
rc = ena_com_phc_get_timestamp(phc_info->adapter->ena_dev,
43
+
×tamp_nsec);
44
+
45
+
ptp_read_system_postts(sts);
46
+
47
+
spin_unlock_irqrestore(&phc_info->lock, flags);
48
+
49
+
*ts = ns_to_timespec64(timestamp_nsec);
50
+
51
+
return rc;
52
+
}
53
+
54
+
static int ena_phc_settime64(struct ptp_clock_info *clock_info,
55
+
const struct timespec64 *ts)
56
+
{
57
+
return -EOPNOTSUPP;
58
+
}
59
+
60
+
static struct ptp_clock_info ena_ptp_clock_info = {
61
+
.owner = THIS_MODULE,
62
+
.n_alarm = 0,
63
+
.n_ext_ts = 0,
64
+
.n_per_out = 0,
65
+
.pps = 0,
66
+
.adjtime = ena_phc_adjtime,
67
+
.adjfine = ena_phc_adjfine,
68
+
.gettimex64 = ena_phc_gettimex64,
69
+
.settime64 = ena_phc_settime64,
70
+
.enable = ena_phc_feature_enable,
71
+
};
72
+
73
+
/* Enable/Disable PHC by the kernel, affects on the next init flow */
74
+
void ena_phc_enable(struct ena_adapter *adapter, bool enable)
75
+
{
76
+
struct ena_phc_info *phc_info = adapter->phc_info;
77
+
78
+
if (!phc_info) {
79
+
netdev_err(adapter->netdev, "phc_info is not allocated\n");
80
+
return;
81
+
}
82
+
83
+
phc_info->enabled = enable;
84
+
}
85
+
86
+
/* Check if PHC is enabled by the kernel */
87
+
bool ena_phc_is_enabled(struct ena_adapter *adapter)
88
+
{
89
+
struct ena_phc_info *phc_info = adapter->phc_info;
90
+
91
+
return (phc_info && phc_info->enabled);
92
+
}
93
+
94
+
/* PHC is activated if ptp clock is registered in the kernel */
95
+
bool ena_phc_is_active(struct ena_adapter *adapter)
96
+
{
97
+
struct ena_phc_info *phc_info = adapter->phc_info;
98
+
99
+
return (phc_info && phc_info->clock);
100
+
}
101
+
102
+
static int ena_phc_register(struct ena_adapter *adapter)
103
+
{
104
+
struct pci_dev *pdev = adapter->pdev;
105
+
struct ptp_clock_info *clock_info;
106
+
struct ena_phc_info *phc_info;
107
+
int rc = 0;
108
+
109
+
phc_info = adapter->phc_info;
110
+
clock_info = &phc_info->clock_info;
111
+
112
+
/* PHC may already be registered in case of a reset */
113
+
if (ena_phc_is_active(adapter))
114
+
return 0;
115
+
116
+
phc_info->adapter = adapter;
117
+
118
+
spin_lock_init(&phc_info->lock);
119
+
120
+
/* Fill the ptp_clock_info struct and register PTP clock */
121
+
*clock_info = ena_ptp_clock_info;
122
+
snprintf(clock_info->name,
123
+
sizeof(clock_info->name),
124
+
"ena-ptp-%02x",
125
+
PCI_SLOT(pdev->devfn));
126
+
127
+
phc_info->clock = ptp_clock_register(clock_info, &pdev->dev);
128
+
if (IS_ERR(phc_info->clock)) {
129
+
rc = PTR_ERR(phc_info->clock);
130
+
netdev_err(adapter->netdev, "Failed registering ptp clock, error: %d\n",
131
+
rc);
132
+
phc_info->clock = NULL;
133
+
}
134
+
135
+
return rc;
136
+
}
137
+
138
+
static void ena_phc_unregister(struct ena_adapter *adapter)
139
+
{
140
+
struct ena_phc_info *phc_info = adapter->phc_info;
141
+
142
+
/* During reset flow, PHC must stay registered
143
+
* to keep kernel's PHC index
144
+
*/
145
+
if (ena_phc_is_active(adapter) &&
146
+
!test_bit(ENA_FLAG_TRIGGER_RESET, &adapter->flags)) {
147
+
ptp_clock_unregister(phc_info->clock);
148
+
phc_info->clock = NULL;
149
+
}
150
+
}
151
+
152
+
int ena_phc_alloc(struct ena_adapter *adapter)
153
+
{
154
+
/* Allocate driver specific PHC info */
155
+
adapter->phc_info = vzalloc(sizeof(*adapter->phc_info));
156
+
if (unlikely(!adapter->phc_info)) {
157
+
netdev_err(adapter->netdev, "Failed to alloc phc_info\n");
158
+
return -ENOMEM;
159
+
}
160
+
161
+
return 0;
162
+
}
163
+
164
+
void ena_phc_free(struct ena_adapter *adapter)
165
+
{
166
+
if (adapter->phc_info) {
167
+
vfree(adapter->phc_info);
168
+
adapter->phc_info = NULL;
169
+
}
170
+
}
171
+
172
+
int ena_phc_init(struct ena_adapter *adapter)
173
+
{
174
+
struct ena_com_dev *ena_dev = adapter->ena_dev;
175
+
struct net_device *netdev = adapter->netdev;
176
+
int rc = -EOPNOTSUPP;
177
+
178
+
/* Validate PHC feature is supported in the device */
179
+
if (!ena_com_phc_supported(ena_dev)) {
180
+
netdev_dbg(netdev, "PHC feature is not supported by the device\n");
181
+
goto err_ena_com_phc_init;
182
+
}
183
+
184
+
/* Validate PHC feature is enabled by the kernel */
185
+
if (!ena_phc_is_enabled(adapter)) {
186
+
netdev_dbg(netdev, "PHC feature is not enabled by the kernel\n");
187
+
goto err_ena_com_phc_init;
188
+
}
189
+
190
+
/* Initialize device specific PHC info */
191
+
rc = ena_com_phc_init(ena_dev);
192
+
if (unlikely(rc)) {
193
+
netdev_err(netdev, "Failed to init phc, error: %d\n", rc);
194
+
goto err_ena_com_phc_init;
195
+
}
196
+
197
+
/* Configure PHC feature in driver and device */
198
+
rc = ena_com_phc_config(ena_dev);
199
+
if (unlikely(rc)) {
200
+
netdev_err(netdev, "Failed to config phc, error: %d\n", rc);
201
+
goto err_ena_com_phc_config;
202
+
}
203
+
204
+
/* Register to PTP class driver */
205
+
rc = ena_phc_register(adapter);
206
+
if (unlikely(rc)) {
207
+
netdev_err(netdev, "Failed to register phc, error: %d\n", rc);
208
+
goto err_ena_com_phc_config;
209
+
}
210
+
211
+
return 0;
212
+
213
+
err_ena_com_phc_config:
214
+
ena_com_phc_destroy(ena_dev);
215
+
err_ena_com_phc_init:
216
+
ena_phc_enable(adapter, false);
217
+
ena_devlink_disable_phc_param(adapter->devlink);
218
+
return rc;
219
+
}
220
+
221
+
void ena_phc_destroy(struct ena_adapter *adapter)
222
+
{
223
+
ena_phc_unregister(adapter);
224
+
ena_com_phc_destroy(adapter->ena_dev);
225
+
}
226
+
227
+
int ena_phc_get_index(struct ena_adapter *adapter)
228
+
{
229
+
if (ena_phc_is_active(adapter))
230
+
return ptp_clock_index(adapter->phc_info->clock);
231
+
232
+
return -1;
233
+
}
+37
drivers/net/ethernet/amazon/ena/ena_phc.h
+37
drivers/net/ethernet/amazon/ena/ena_phc.h
···
1
+
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
2
+
/*
3
+
* Copyright 2015-2022 Amazon.com, Inc. or its affiliates. All rights reserved.
4
+
*/
5
+
6
+
#ifndef ENA_PHC_H
7
+
#define ENA_PHC_H
8
+
9
+
#include <linux/ptp_clock_kernel.h>
10
+
11
+
struct ena_phc_info {
12
+
/* PTP hardware capabilities */
13
+
struct ptp_clock_info clock_info;
14
+
15
+
/* Registered PTP clock device */
16
+
struct ptp_clock *clock;
17
+
18
+
/* Adapter specific private data structure */
19
+
struct ena_adapter *adapter;
20
+
21
+
/* PHC lock */
22
+
spinlock_t lock;
23
+
24
+
/* Enabled by kernel */
25
+
bool enabled;
26
+
};
27
+
28
+
void ena_phc_enable(struct ena_adapter *adapter, bool enable);
29
+
bool ena_phc_is_enabled(struct ena_adapter *adapter);
30
+
bool ena_phc_is_active(struct ena_adapter *adapter);
31
+
int ena_phc_get_index(struct ena_adapter *adapter);
32
+
int ena_phc_init(struct ena_adapter *adapter);
33
+
void ena_phc_destroy(struct ena_adapter *adapter);
34
+
int ena_phc_alloc(struct ena_adapter *adapter);
35
+
void ena_phc_free(struct ena_adapter *adapter);
36
+
37
+
#endif /* ENA_PHC_H */
+8
drivers/net/ethernet/amazon/ena/ena_regs_defs.h
+8
drivers/net/ethernet/amazon/ena/ena_regs_defs.h
···
53
53
#define ENA_REGS_MMIO_RESP_HI_OFF 0x64
54
54
#define ENA_REGS_RSS_IND_ENTRY_UPDATE_OFF 0x68
55
55
56
+
/* phc_registers offsets */
57
+
58
+
/* 100 base */
59
+
#define ENA_REGS_PHC_DB_OFF 0x100
60
+
56
61
/* version register */
57
62
#define ENA_REGS_VERSION_MINOR_VERSION_MASK 0xff
58
63
#define ENA_REGS_VERSION_MAJOR_VERSION_SHIFT 8
···
133
128
#define ENA_REGS_RSS_IND_ENTRY_UPDATE_INDEX_MASK 0xffff
134
129
#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_SHIFT 16
135
130
#define ENA_REGS_RSS_IND_ENTRY_UPDATE_CQ_IDX_MASK 0xffff0000
131
+
132
+
/* phc_db_req_id register */
133
+
#define ENA_REGS_PHC_DB_REQ_ID_MASK 0xffff
136
134
137
135
#endif /* _ENA_REGS_H_ */
+4
include/net/devlink.h
+4
include/net/devlink.h
···
520
520
DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP,
521
521
DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE,
522
522
DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE,
523
+
DEVLINK_PARAM_GENERIC_ID_ENABLE_PHC,
523
524
524
525
/* add new param generic ids above here*/
525
526
__DEVLINK_PARAM_GENERIC_ID_MAX,
···
578
577
579
578
#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME "event_eq_size"
580
579
#define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32
580
+
581
+
#define DEVLINK_PARAM_GENERIC_ENABLE_PHC_NAME "enable_phc"
582
+
#define DEVLINK_PARAM_GENERIC_ENABLE_PHC_TYPE DEVLINK_PARAM_TYPE_BOOL
581
583
582
584
#define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \
583
585
{ \
+5
net/devlink/param.c
+5
net/devlink/param.c
···
92
92
.name = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME,
93
93
.type = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE,
94
94
},
95
+
{
96
+
.id = DEVLINK_PARAM_GENERIC_ID_ENABLE_PHC,
97
+
.name = DEVLINK_PARAM_GENERIC_ENABLE_PHC_NAME,
98
+
.type = DEVLINK_PARAM_GENERIC_ENABLE_PHC_TYPE,
99
+
},
95
100
};
96
101
97
102
static int devlink_param_generic_verify(const struct devlink_param *param)