Documentation Troubleshooting

CHAPTER 25

Troubleshooting

10 sections · 20 min read · SimpleVM 10.1

Part XII — Troubleshooting

When something goes wrong on a SimpleVM host, the instinct is to start changing settings until the symptom disappears. That instinct is what turns a small problem into a large one. The reliable path is the opposite: slow down, describe the symptom precisely, find out what changed, and make one corrective change at a time — keeping a way back at every step. This chapter exists so you have that method in hand and a place to jump to for each kind of problem, instead of hunting through the rest of the guide while a VM is down.

The chapter is organized by symptom domain. If you already know the area — storage, networking, an update that failed — go straight to that section. If you do not, start with Section 25.1, Troubleshooting Methodology, which gives you the sequence and the four questions that narrow almost any problem to its real cause. Every domain section follows the same shape: a short explanation of where the fault usually lives, a symptom-to-first-check table, and — where a corrective action changes the host — how to undo it.

25.1 Troubleshooting Methodology

Overview

Troubleshooting is most effective when it is structured. A method does two things at once: it keeps you from making random changes that create new problems, and it preserves evidence before the environment changes again. The goal of every step below is to narrow the problem — to shrink the set of possible causes until one change, made deliberately, resolves the symptom.

The most important habit is to change one thing at a time and verify it. Reverting the last change you made is usually safer than adding another change on top of it, and it is almost always faster than rebuilding from a state you can no longer explain.

The troubleshooting sequence

Work through these steps in order. Resist the urge to skip ahead to a fix before you have defined the symptom and its scope:

  • Define the symptom in plain language. Write down what is actually slow, failing, or missing — not your guess at the cause.
  • Identify the scope. Is it one VM, several VMs, the host, storage, network, firewall, updates, migration, or cluster behavior? Scope separates a guest problem from a host-wide one.
  • Identify when it started. A precise start time often lines up with a change.
  • List recent changes. Updates, configuration edits, new workloads, storage or network changes, firewall edits, migrations, or hardware work — anything from before the symptom appeared.
  • Check the simplest visible status first. Open the web console and read state — health, VM status, pool and network status — before reaching for the command line.
  • Collect only the information you need for the suspected area. Gather evidence; do not start changing things yet.
  • Make one corrective change at a time.
  • Verify the result. Confirm the original symptom is gone and that no new problem appeared.
  • Document the cause and the fix so the next person — possibly you — does not start from zero.

The four anchor questions

When a problem resists the sequence, return to these four questions. They re-orient almost any investigation:

QuestionWhy it matters
What changed?Most problems begin after a configuration, update, workload, storage, network, firewall, or hardware change. Find the change and you usually find the cause.
What is affected?Scope separates a single-guest problem from a host-wide one and points you at the right domain section.
What still works?The working areas narrow the fault domain as effectively as the broken ones do.
Can it be safely reverted?Reverting the last known change is often safer than adding more changes. If a change cannot be cleanly reverted, that is itself important to know before you proceed.

Verification

You are ready to act when you can state the symptom in one sentence, name its scope, point to the most recent relevant change, and describe the single corrective change you intend to make and how you will undo it. If you cannot, you are not yet troubleshooting — you are guessing. Return to the sequence and finish narrowing first.

Revert this change

This section establishes a method and changes nothing on its own. The reversibility requirement it describes applies to every corrective action in the domain sections that follow: before you change a firewall rule, a network setting, an update state, or a storage object, record the current value so the revert step has an exact target. Where a change cannot be cleanly reverted — most often in storage — the relevant section says so and gives the safest recovery path instead.

25.2 Installation Issues

Overview

Installation problems almost always come down to one of five things: the installation media, the boot mode, the storage selection, the network configuration, or hardware compatibility. This section gives first-pass checks; the full installation workflow lives in Chapter 4, Installing SimpleVM, which is the right place to confirm each step in detail.

Figure 25-1a. The Installation Summary screen, reviewed before installation begins. Confirm the installation destination here so a data disk is never selected by mistake. The installer is covered in Chapter 4, Installing SimpleVM.
Figure 25-1a. The Installation Summary screen, reviewed before installation begins. Confirm the installation destination here so a data disk is never selected by mistake. The installer is covered in Chapter 4, Installing SimpleVM.

Common symptoms and first checks

SymptomLikely causeFirst check
Installer does not bootBoot order, media, or firmware mode.Confirm boot order, that the install media is present, the UEFI/BIOS mode, and that the ISO was written correctly.
Target disk is not availableStorage controller or RAID not presenting the disk.Confirm the storage controller, the RAID virtual disk, and drive visibility in the server firmware.
Installation destination warningThe wrong disk may be selected.Review the selected disks and confirm the intended OS disk. Do not select a data disk by mistake.
No network during installationCabling, switch port, VLAN, or addressing.Confirm the cable, switch port, VLAN, DHCP or static settings, and the hostname configuration.
First boot failsThe host is still booting from the install media.Confirm the server boots from the installed OS disk, not the installation media.

Verification

After installation, the host should boot to the SimpleVM host console and display the management URLs and a login prompt. If it does not, confirm boot device order and the installation destination before anything else.

25.3 Login and Access Issues

Overview

Access problems can sit at any of four layers: the browser, the network between you and the host, the host's services, or your credentials and privilege level. The fastest way to triage is to start at the host console, because it tells you whether the host is up and what address and port it expects you to use — even when the browser cannot reach it. The management interface is the web console on TCP port 9090.

Before you begin

  • Physical or remote console access to the host (so you can read the console when the browser fails).
  • The management URL and the host's IP address, both shown on the host console after boot.
  • A supported browser on an administrator workstation that can reach the host network.
Figure 25-1. The host console after boot. It confirms the product and build (Thinware SimpleVM 10.1.2601.1), the Management Console URLs, and the port (9090), and presents the login prompt. When the browser cannot reach the interface, this screen tells you whether the host is up and what address to use. The console is introduced in Chapter 6, Accessing the SimpleVM Management Interface.
Figure 25-1. The host console after boot. It confirms the product and build (Thinware SimpleVM 10.1.2601.1), the Management Console URLs, and the port (9090), and presents the login prompt. When the browser cannot reach the interface, this screen tells you whether the host is up and what address to use. The console is introduced in Chapter 6, Accessing the SimpleVM Management Interface.

Procedure: triage access from the console

  1. Confirm the SimpleVM host is powered on and reachable on the network.
  2. At the host console, read the displayed management URL and IP address.
  3. From an administrator workstation, open the management URL in a supported browser.
  4. Confirm the browser is using the correct protocol, host name or IP address, and the 9090 port.
  5. If the page does not load, move to host networking and firewall (Section 25.6, Network Issues, and Section 25.7, Firewall Issues).
  6. If the page loads but login fails, confirm the user name, password, account status, and that the account has administrative privileges (Chapter 17, User Access and Administrative Tasks).

Common symptoms and first checks

SymptomLikely causeFirst check
Browser cannot reach the web consoleNetwork path, addressing, or firewall.Confirm the IP address, DNS, gateway, and firewall, and that the host is online. Triage from the console first.
Certificate warning appearsExpected for the current certificate configuration, or a real mismatch.Confirm whether the warning is expected for the current certificate setup. — confirm certificate behavior.
Login failsCredentials, keyboard layout, or privilege.Confirm credentials, keyboard layout, account status, and administrative privileges.
Session disconnectsNetwork stability or host load.Check network stability, browser behavior, and host load (Section 25.6; Chapter 23, Monitoring Your SimpleVM Host).

Revert this change

Triage in this section is read-only. If, while testing, you changed a firewall rule (Chapter 16, Firewall and Security Settings) or a host network setting (Chapter 15, Managing Host Networking) to restore access, restore the original setting from its home chapter once access is confirmed, and verify you can still reach the console afterward. Do not leave a broad temporary firewall rule in place after the problem is resolved.

25.4 VM Creation and Boot Issues

Overview

A VM that will not create or will not boot is usually held up by something it depends on rather than by the VM itself: missing installation media, an inactive storage pool, not enough host resources, a wrong boot order, an incompatible guest setting, or a problem inside the guest operating system. Check the things the VM depends on before you change the VM.

Procedure: narrow a VM that will not create or boot

  1. Open Virtual machines and select the affected VM.
  2. Review the VM state and open its Console.
  3. Review its attached disks, network interfaces, and installation media (Chapter 9, Managing VM Disks; Chapter 10, Managing VM Network Interfaces).
  4. Confirm the target storage pool is active and has free capacity (Chapter 12, Managing Storage Pools).
  5. Confirm the virtual network is active and matches the intended design (Chapter 14, Managing Virtual Networks).
  6. If the VM failed after a configuration change, revert the most recent change first, then re-test.

Common symptoms and first checks

SymptomLikely causeFirst check
Create VM action failsHost resources, pool state, ISO, or a required field.Check host resources, storage pool status, ISO availability, and required fields in the create workflow.
VM does not startInsufficient resources or a transitional state.Check available CPU, memory, and storage, and whether the VM is already in a transitional state.
VM boots to the wrong deviceAttached ISO or boot order.Check the attached ISO, boot order, and virtual disk configuration.
Guest installer cannot see the diskDisk not attached or driver missing.Confirm the disk is attached and that the guest has the required storage driver.
Guest installer has no networkInterface or virtual-network selection.Confirm the VM network interface and the selected virtual network.
VM console is blankStill booting, or display settings.Wait for boot, refresh the console, check power state, and confirm guest display settings.

Revert this change

If the VM stopped working after an edit, the safest first move is to undo that edit before trying anything new: restore the previous CPU, memory, disk, interface, or boot setting from the VM detail page (Chapter 8, Creating and Managing Virtual Machines). If the VM has a snapshot from before the change, reverting to it is a clean way back — but a snapshot is not a backup and depends on the same host storage. See Chapter 11, VM Snapshots.

25.5 Storage Issues

Overview

Storage problems show up almost everywhere — they can block VM creation, VM startup, ISO uploads, snapshots, migration, and backups. The discipline here is strict: always confirm storage pool state and capacity first, and treat destructive actions as a last resort, never a first response. A full pool imitates many other problems, so capacity is the very first thing to rule out.

Common symptoms and first checks

SymptomLikely causeFirst check
Storage pool is unavailablePool inactive, or its path or device is missing.Confirm whether the pool is active and whether the underlying path or device is present (Chapter 12).
Storage pool is fullCapacity exhausted.Remove unneeded ISOs only after confirmation, expand storage, or migrate workloads to another pool.
VM disk is missingDisk removed, deleted, or in another pool.Confirm whether the disk was removed from the VM, deleted from storage, or located in a different pool (Chapter 9).
Snapshot cannot be createdFree space or snapshot support for the disk type.Check free space and snapshot support for the VM disk and storage type.
Host storage shows unexpected disksRAID, external storage, or drive replacement history.Confirm server RAID, external storage, and drive replacement history before making any change (Chapter 13, Managing Host Storage).

Revert this change

Storage changes are the one place in this chapter where “just undo it” may not apply. If a VM disk, storage pool, logical volume, or partition has been deleted or overwritten, the safest recovery path is usually backup restoration or support-assisted recovery — not another change. Stop writing to the affected storage until you have reviewed your recovery options. See Chapter 20, Backup and Recovery.

25.6 Network Issues

Overview

Network problems are easiest to solve when you narrow them by layer rather than guessing. Work outward from the host: does the SimpleVM host itself have network access? Is the virtual network active? Does the VM have an interface attached to it? And finally, is the real problem outside SimpleVM entirely, on the physical switch or upstream network? Each layer you confirm removes a large set of possible causes.

Procedure: narrow a network problem by layer

  1. Confirm the SimpleVM host has network connectivity (Chapter 15, Managing Host Networking).
  2. Confirm the affected VM has a network interface attached (Chapter 10, Managing VM Network Interfaces).
  3. Confirm the virtual network is active (Chapter 14, Managing Virtual Networks).
  4. Check whether other VMs on the same virtual network work — that isolates a single-VM problem from a network-wide one.
  5. If traffic is blocked, review firewall zones and allowed services (Section 25.7, Firewall Issues).
  6. If a bridge or physical interface changed recently, compare the current configuration with the previous known-good one before changing it again.

Common symptoms and first checks

SymptomLikely causeFirst check
Host is unreachablePhysical link, addressing, VLAN, or firewall.Check physical link, IP address, gateway, DNS, VLAN, and firewall.
VM has no network accessInterface, virtual-network state, or guest IP.Check the VM interface, virtual-network status, DHCP or static settings, and guest IP configuration.
VM reaches host but not outsideNAT/bridge mode, gateway, DNS, or upstream path.Check NAT or bridge mode, gateway, DNS, firewall, and the physical network path.
Only one VM is affectedThat VM's interface or guest settings.Compare its interface and guest IP settings with a working VM.
All VMs on one network are affectedThe virtual network or a recent host change.Review the virtual network and any recent host networking change.

Revert this change

If you changed a VM's virtual network attachment, a host bridge or VLAN, or a firewall rule to test a hypothesis, restore the original setting from its home chapter — virtual network attachment in Chapter 14, Managing Virtual Networks, host bridges, bonds, and VLANs in Chapter 15, Managing Host Networking, and firewall rules in Chapter 16, Firewall and Security Settings — and confirm connectivity returns before moving on.

25.7 Firewall Issues

Overview

Firewall problems wear many disguises: blocked web console access, VM communication that fails, a migration or cluster operation that times out, or a service that works locally but not from another system. Before assuming an application is broken, review the firewall zones and the services they allow. The firewall is zone-based, and the default zones each serve a purpose — keep it enabled and confirm the required communication works, rather than turning it off.

Figure 25-2. The firewall Zones detail showing the public, cluster, and libvirt zones and their allowed services. Web console access (port 9090) is in the public zone; SSH and the migration port range are in the cluster zone; DHCP and DNS for virtual networks are in the libvirt zone. Firewall configuration is covered in Chapter 16, Firewall and Security Settings.
Figure 25-2. The firewall Zones detail showing the public, cluster, and libvirt zones and their allowed services. Web console access (port 9090) is in the public zone; SSH and the migration port range are in the cluster zone; DHCP and DNS for virtual networks are in the libvirt zone. Firewall configuration is covered in Chapter 16, Firewall and Security Settings.

Procedure: review and test firewall behavior

  1. Open the firewall Zones page.
  2. Review the active zones.
  3. Confirm the expected interface or source is assigned to the correct zone.
  4. Review the allowed services and ports for that zone.
  5. Compare the current state with the communication you expect to work — web console, VM network, migration, or cluster traffic.
  6. Make only the minimum rule change required to test the issue, then verify.

Common symptoms and first checks

SymptomLikely causeFirst check
Web console unreachable from a workstationPort 9090 not allowed on the active zone.Confirm the public zone allows the web console and that the workstation's path reaches it (Section 25.3).
Migration or cluster operation failsSSH or the migration port range is blocked.Confirm the cluster zone allows SSH and the migration port range on both hosts (Chapter 22).
VM network traffic blockedDHCP/DNS or zone assignment for virtual networks.Review the libvirt zone services and the virtual network (Chapter 14).
Service works locally but not remotelyA zone or rule is filtering the remote source.Compare the active zone's allowed services against the required service and source.

Revert this change

If a firewall change does not resolve the issue, remove the rule or restore the previous zone and service assignment immediately. Avoid leaving broad temporary access rules in place after troubleshooting — a rule added “just to test” and forgotten is a standing security gap. Reload the firewall after restoring the original state and confirm the expected traffic still behaves as before.

25.8 Update Issues

Overview

Update problems usually involve unreachable package repositories, an interrupted update transaction, insufficient disk space, a pending reboot, or a service that needs to restart after the update completes. The Software updates page is the first checkpoint — it shows the status, the settings, and the history, which together explain most update problems without touching the command line.

Figure 25-3. The Software updates page showing system status, update settings, and update history. Start update troubleshooting here. The update workflow is covered in Chapter 18, Software Updates.
Figure 25-3. The Software updates page showing system status, update settings, and update history. Start update troubleshooting here. The update workflow is covered in Chapter 18, Software Updates.

Procedure: triage a failed or missing update

  1. Open Software updates.
  2. Review the update status and update history.
  3. Check whether a reboot is required.
  4. Confirm the host has network and DNS access (Section 25.6).
  5. Confirm storage is not full (Section 25.5).
  6. If an update failed, record the visible error before retrying.
  7. Use the Terminal only if the update page does not provide enough detail (Chapter 19, Host Administration Tools).

Common symptoms and first checks

SymptomLikely causeFirst check
Updates do not appearNetwork, DNS, repository access, or settings.Check network access, DNS, repositories, and update settings. — confirm the official update and repository model.
Update failsCapacity, network, or transaction state.Check storage capacity, network access, the package transaction status, and the visible error message.
Reboot requiredUpdated components need a restart.Schedule a maintenance window and reboot the host if production workloads are affected.
Issue appears after an updateA changed component affects a service or VM.Review the update history and determine whether the issue affects host services, VMs, storage, or networking.

Revert this change

Updates are not a simple toggle to undo. If a problem appears after an update, use the update history to identify what changed, and plan a recovery around it rather than reapplying changes blindly: the safe path is usually to roll the host back to a known-good state from backup, or to follow the supported recovery procedure, after confirming the update is the cause. Confirm the official update and rollback workflow with support before relying on a manual reversal. See Chapter 18, Software Updates, and Chapter 20, Backup and Recovery. — official update and rollback workflow.

25.9 Migration and Cluster Issues

Overview

Migration and cluster problems pull in more moving parts than any other domain: host reachability, shared storage, CPU compatibility, firewall rules, VM state, authentication between hosts, network reliability, and cluster configuration. Start from the migration and cluster requirements already documented in Chapter 21, VM Migration, and Chapter 22, Configuring Host Clusters for Live VM Migration and High Availability — most failures trace back to a requirement that is not met on one of the hosts involved.

Common symptoms and first checks

SymptomLikely causeFirst check
Live migration failsReachability, state, storage, firewall, or CPU.Check source and target reachability, VM state, storage access, firewall rules, and CPU compatibility.
Migration is slowNetwork, storage, or guest memory activity.Check network utilization, storage performance, VM memory activity, and background workload (Chapter 24).
VM cannot run on the target hostCapacity, storage, network, or CPU on the target.Check target host capacity, storage availability, virtual network availability, and CPU compatibility.
Cluster node appears unavailablePower, network, firewall, time, or service state.Check host power, network, firewall, time synchronization, and cluster service status.
High availability action does not occurHA configuration, shared storage, or quorum.Check HA configuration, shared storage, cluster quorum, and support status.

Procedure: narrow a migration or cluster failure

  1. Confirm both hosts are online and reachable from each other.
  2. Confirm the required storage is available to every host involved.
  3. Confirm the required virtual networks exist on the target host.
  4. Review firewall rules and the cluster and migration requirements (Section 25.7; Chapter 22).
  5. Check whether the affected VM has special CPU, memory, storage, or device configuration that limits migration.
  6. Retry only after the cause of the previous failure has been identified or changed — not before.

Revert this change

A failed migration usually leaves the VM where it started; confirm its state on the source host before retrying. If you changed firewall rules, SSH access between hosts, or cluster settings while troubleshooting, restore the original configuration from Chapter 16, Firewall and Security Settings, and Chapter 22, Configuring Host Clusters for Live VM Migration and High Availability, and verify the cluster returns to a healthy state before attempting the operation again.

25.10 Collecting Information for Support

Overview

When an issue cannot be resolved through normal troubleshooting, the next step is to collect clear information before contacting support. Good information reduces back-and-forth, helps preserve the state of the problem, and lets support help you on the first exchange instead of the third. Gather the picture first; gather logs only as needed.

Support information checklist

Collect the following before you open a support request:

  • SimpleVM version and host name. — confirm the official version command.
  • Affected VM name, guest operating system, and VM state.
  • The time the issue started and whether it is still occurring.
  • Recent changes — updates, storage, network, or firewall changes, VM edits, migration attempts, or cluster maintenance.
  • Screenshots of the visible error or the affected page.
  • Scope — whether the issue affects one VM, multiple VMs, the host, storage, networking, updates, migration, or clustering.
  • Steps already attempted, and whether each step changed the behavior.
  • Any logs or command output that support requests.
Figure 25-4. The Terminal page, used to collect read-only information when the web console does not show enough. It opens as the signed-in administrative user (the [svmadmin@tw-rnd1 /]$ prompt), not as root. The web terminal and the related Logs and Diagnostic reports tools are covered in Chapter 19, Host Administration Tools.
Figure 25-4. The Terminal page, used to collect read-only information when the web console does not show enough. It opens as the signed-in administrative user (the [svmadmin@tw-rnd1 /]$ prompt), not as root. The web terminal and the related Logs and Diagnostic reports tools are covered in Chapter 19, Host Administration Tools.

Read-only first-pass commands

Use the Terminal or SSH only when you need information that is not visible in the web console. The commands below are first-pass, read-only information-gathering examples — they observe state and change nothing. They are not a replacement for the official support guidance:

hostnamectl
uptime
df -h
free -h
ip addr
systemctl --failed
journalctl -p warning..alert -n 100
virsh list --all

Each line reports state without altering it: the host name and identity, how long the host has been up, filesystem and memory usage, assigned IP addresses, any failed services, recent high-priority log entries, and the list of all VMs and their state. Run them in this order to build the picture support will ask for. Commands and log locations should be verified against the final SimpleVM support process before they are published as an official procedure.

Revert this change

Nothing in this section changes host state — the checklist is documentation and the first-pass commands are read-only, so there is nothing to revert. This is the intended end state of a troubleshooting session that could not be resolved locally: you have preserved the evidence, made no further changes, and handed support a complete picture.