How do I troubleshoot iSCSI storage performance issues?

How do I troubleshoot iSCSI storage performance issues?

Symptoms

When you encounter an issue with iSCSI storage performance, we encourage you to perform basic troubleshooting with the instructions in this article. There are several possible issues you may encounter:

  • Low throughput
  • High latency
  • Other storage, system, or network issues

Resolution

To troubleshoot a low throughput:

If you think the throughput is lower than expected when checking Windows File Explorer, we suggest that you try different benchmarking tools and refer to the information below.

Understanding your benchmark index:

The performance tests of Synology NAS products are conducted using IOMeter as the benchmarking tool and SSDs as the storage media, and multiple workers and threads are running to maximize the throughput of the iSCSI storage. If you use Windows File Explorer to copy a file from one location to another, the speed displayed at the Windows UI may not reflect the actual capability of the iSCSI storage.

To troubleshoot a high latency:

If your applications (virtual machines or application servers) experience a high latency (larger than 30 ms in general), follow the steps below to identify the root cause:

  1. Go to DSM > Resource Monitor > Performance > iSCSI (For DSM 7.0, the tab is renamed as LUN).
  2. Click Choose Charts.
  3. Check Disk Latency and Network Latency, and click OK.
  4. You will then discover what contributes to the iSCSI storage latency. Usually it is the disk.
    • If the disk latency is too high, please go through Checklist 1 to check the storage status.
    • If the network latency is too high, please go through Checklist 2 to check the iSCSI network environment.

To troubleshoot issues with the storage, system, or network:

Go through the following checklists to identify possible root causes of compromised performance.

Checklist 1: Storage and system

Item
What to do
Why
RAID/volume health
Instead doing any rebuilding, repairing, or expanding in Storage Manager, check if the status of each of the following items is normal and in good health:
  • SSD Cache
  • RAID Group
  • Volume
  1. After you create a new data volume, the system might spend some time inspecting it, which will consume the system resources. The Synology NAS will only be able to dedicate full resources to iSCSI services when it has finished checking the new volume.
  2. When Read/Write SSD cache is degraded, the performance will be degraded as well since no write operations will go through SSDs.
HDD/SSD health
Check if the HDDs or SSDs are healthy at Storage Manager > HDD/SDD.
Defective drives might hinder the full utilization of the storage system.
System status
Examine the CPU and disk utilization in Resource Monitor, or disable all the services except for the ones that you would like to run at Control Panel > Info Center > Service.
To make sure no resource-intensive processes are running on your Synology NAS.

Checklist 2: Storage and network

Item
What to do
Why
Multipath I/O settings
Make sure MPIO is set up properly on your hosts. Refer to the following tutorials:
It is recommended that you configure MPIO instead of network bonding (or Link Aggregation) to maximize the throughput for all hosts. Also, the stability and performance of MPIO configuration have been thoroughly verified before product and software update releases.
iSCSI network
Make sure the iSCSI network is isolated and is a dedicated VLAN. Isolate iSCSI traffic from other traffics, such as file transfer or virtual machines.
Mixing the iSCSI traffic with other network traffics is not recommended because it will complicate the troubleshooting process with additional unknown factors.
iSCSI HBA
If you are using a hardware iSCSI initiator, try software iSCSI initiators and compare the performances.
Our iSCSI storage service is verified with the following iSCSI initiators:
  • vSphere ESXi iSCSI Software initiator HBA
  • Microsoft software iSCSI initiator
  • XenServer software iSCSI initiator

Symptoms
Resolution
To troubleshoot a low throughput:
To troubleshoot a high latency:
To troubleshoot issues with the storage, system, or network: