Discussion:
[libvirt] [PATCH 0/2] qemu: fix misc noisy logs for optional sources
Nikolay Shirokovskiy
2018-11-12 12:58:17 UTC
Permalink
Nikolay Shirokovskiy (2):
qemu: don't log error for missing optional sources on stats
qemu: don't log error for missing optional sources on start

src/qemu/qemu_driver.c | 5 +++++
src/qemu/qemu_process.c | 12 +++++++++++-
src/qemu/qemu_process.h | 2 ++
3 files changed, 18 insertions(+), 1 deletion(-)
--
1.8.3.1
Nikolay Shirokovskiy
2018-11-12 12:58:19 UTC
Permalink
Because missing optional source is not error. The patch
address only local files. Fixing other cases is a bit ugly.

Signed-off-by: Nikolay Shirokovskiy <***@virtuozzo.com>
---
src/qemu/qemu_process.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 802274e..5e04bf9 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6100,7 +6100,8 @@ qemuProcessPrepareHostStorage(virQEMUDriverPtr driver,
if (!blockdev)
virStorageSourceBackingStoreClear(disk->src);

- if (qemuDomainDetermineDiskChain(driver, vm, disk, true) >= 0)
+ if (!qemuProcessMissingLocalOptionalDisk(disk) &&
+ qemuDomainDetermineDiskChain(driver, vm, disk, true) >= 0)
continue;

if (qemuDomainCheckDiskStartupPolicy(driver, vm, idx, cold_boot) >= 0)
--
1.8.3.1
John Ferlan
2018-12-10 22:05:36 UTC
Permalink
$SUBJ

"storage sources"
Post by Nikolay Shirokovskiy
Because missing optional source is not error. The patch
address only local files. Fixing other cases is a bit ugly.
---
src/qemu/qemu_process.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 802274e..5e04bf9 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6100,7 +6100,8 @@ qemuProcessPrepareHostStorage(virQEMUDriverPtr driver,
if (!blockdev)
virStorageSourceBackingStoreClear(disk->src);
- if (qemuDomainDetermineDiskChain(driver, vm, disk, true) >= 0)
+ if (!qemuProcessMissingLocalOptionalDisk(disk) &&
+ qemuDomainDetermineDiskChain(driver, vm, disk, true) >= 0)
Although it makes more sense for this path to use the startupPolicy, I
will point out that the first thing qemuDomainDetermineDiskChain does is
filter on virStorageSourceIsEmpty, so regardless of whether the
startupPolicy is optional or not, not much is happening for empty
sources. So in essence unnecessary at least from my read.

John
Post by Nikolay Shirokovskiy
continue;
if (qemuDomainCheckDiskStartupPolicy(driver, vm, idx, cold_boot) >= 0)
Nikolay Shirokovskiy
2018-11-12 12:58:18 UTC
Permalink
Every time we call all domain stats for inactive domain with
unavailable source we get error message in logs. It's a bit noisy.
While it's arguable whether we need such message or not for mandatory
disks we would like not to see messages for optional disks. Let's
filter at least for cases of local files. Fixing other cases would
require passing flag down the stack to .backendInit of storage
which is ugly.

Stats for active domain are fine because we either drop disks
with unavailable sources or clean source which is handled
by virStorageSourceIsEmpty in qemuDomainGetStatsOneBlockFallback.

We have these logs for successful stats since 25aa7035d which
in turn fixes 596a13713 which added substantial stats for offline
disks.

Signed-off-by: Nikolay Shirokovskiy <***@virtuozzo.com>
---
src/qemu/qemu_driver.c | 5 +++++
src/qemu/qemu_process.c | 9 +++++++++
src/qemu/qemu_process.h | 2 ++
3 files changed, 16 insertions(+)

diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index a52e249..72e4cfe 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20290,6 +20290,11 @@ qemuDomainGetStatsBlockExportDisk(virDomainDiskDefPtr disk,
const char *backendstoragealias;
int ret = -1;

+ if (!virDomainObjIsActive(dom) &&
+ qemuProcessMissingLocalOptionalDisk(disk))
+ return qemuDomainGetStatsBlockExportHeader(disk, disk->src, *recordnr,
+ records, nrecords);
+
for (n = disk->src; virStorageSourceIsBacking(n); n = n->backingStore) {
if (blockdev) {
frontendalias = QEMU_DOMAIN_DISK_PRIVATE(disk)->qomName;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 9cf9718..802274e 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6070,6 +6070,15 @@ qemuProcessPrepareSEVGuestInput(virDomainObjPtr vm)
}


+bool
+qemuProcessMissingLocalOptionalDisk(virDomainDiskDefPtr disk)
+{
+ return disk->startupPolicy == VIR_DOMAIN_STARTUP_POLICY_OPTIONAL &&
+ virStorageSourceIsLocalStorage(disk->src) && disk->src->path &&
+ !virFileExists(disk->src->path);
+}
+
+
static int
qemuProcessPrepareHostStorage(virQEMUDriverPtr driver,
virDomainObjPtr vm,
diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h
index 2037467..52d396d 100644
--- a/src/qemu/qemu_process.h
+++ b/src/qemu/qemu_process.h
@@ -214,4 +214,6 @@ int qemuProcessStartManagedPRDaemon(virDomainObjPtr vm);

void qemuProcessKillManagedPRDaemon(virDomainObjPtr vm);

+bool qemuProcessMissingLocalOptionalDisk(virDomainDiskDefPtr disk);
+
#endif /* __QEMU_PROCESS_H__ */
--
1.8.3.1
John Ferlan
2018-12-10 22:05:19 UTC
Permalink
$SUBJ:

'storage sources'
Post by Nikolay Shirokovskiy
Every time we call all domain stats for inactive domain with
unavailable source we get error message in logs. It's a bit noisy.
Are there ones in particular?
Post by Nikolay Shirokovskiy
While it's arguable whether we need such message or not for mandatory
disks we would like not to see messages for optional disks. Let's
Filtering for the 'startupPolicy = optional' for domain stats (with
empty/unavailable local source) would seem to be a "new use" perhaps not
totally in line with the original intention.
Post by Nikolay Shirokovskiy
filter at least for cases of local files. Fixing other cases would
require passing flag down the stack to .backendInit of storage
which is ugly.
Yeah, I remember chasing down into backendInit (virStorageFileInitAs)
from qemuDomainStorageOpenStat for

virStorageFileBackendForType:145 : internal error: missing storage
backend for 'none' storage
virStorageFileBackendForType:141 : internal error: missing storage
backend for network files using iscsi protocol

where the 'none' comes from a volume using a storage pool of iSCSI
disks. I chased for a bit, but yes it got messy fast.
Post by Nikolay Shirokovskiy
Stats for active domain are fine because we either drop disks
with unavailable sources or clean source which is handled
by virStorageSourceIsEmpty in qemuDomainGetStatsOneBlockFallback.
We have these logs for successful stats since 25aa7035d which
in turn fixes 596a13713 which added substantial stats for offline
disks.
---
src/qemu/qemu_driver.c | 5 +++++
src/qemu/qemu_process.c | 9 +++++++++
src/qemu/qemu_process.h | 2 ++
3 files changed, 16 insertions(+)
diff --git a/src/qemu/qemu_driver.c b/src/qemu/qemu_driver.c
index a52e249..72e4cfe 100644
--- a/src/qemu/qemu_driver.c
+++ b/src/qemu/qemu_driver.c
@@ -20290,6 +20290,11 @@ qemuDomainGetStatsBlockExportDisk(virDomainDiskDefPtr disk,
const char *backendstoragealias;
int ret = -1;
+ if (!virDomainObjIsActive(dom) &&
+ qemuProcessMissingLocalOptionalDisk(disk))
+ return qemuDomainGetStatsBlockExportHeader(disk, disk->src, *recordnr,
+ records, nrecords);
+
From my quick read this part would seem reasonable since the *Frontend
and *Backend wouldn't have valid data and *GetStatsOneBlock is fetching
source sizing data which for an empty source would be unobtainable.
Post by Nikolay Shirokovskiy
for (n = disk->src; virStorageSourceIsBacking(n); n = n->backingStore) {
if (blockdev) {
frontendalias = QEMU_DOMAIN_DISK_PRIVATE(disk)->qomName;
diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c
index 9cf9718..802274e 100644
--- a/src/qemu/qemu_process.c
+++ b/src/qemu/qemu_process.c
@@ -6070,6 +6070,15 @@ qemuProcessPrepareSEVGuestInput(virDomainObjPtr vm)
}
+bool
+qemuProcessMissingLocalOptionalDisk(virDomainDiskDefPtr disk)
Curious why you chose qemu_process and not qemu_domain or even
virstoragefile? This has nothing to do with whether the qemu process is
running or not.
Post by Nikolay Shirokovskiy
+{
+ return disk->startupPolicy == VIR_DOMAIN_STARTUP_POLICY_OPTIONAL &&
While I agree in principle about optional disks; however, since
startupPolicy is optional it would seem this particular check is chasing
a very specific condition. Would it be reasonable to use a flag instead?
Something passed on the command line to essentially say to only collect
the name/format the name of the local empty sources.

That way we're not reusing something that has other uses. Although I'm
open to other opinions.
Post by Nikolay Shirokovskiy
+ virStorageSourceIsLocalStorage(disk->src) && disk->src->path &&
Use virStorageSourceIsEmpty instead.
Post by Nikolay Shirokovskiy
+ !virFileExists(disk->src->path);
And while I believe I understand why you chose this, it would seem to me
that it might be useful to know if an optional local disk had a path
disappear. IOW: If all the other conditions are met, I'd like to know
that the source path is missing. That might be a good thing to know if
I'm getting stats for a domain. Maybe that's just me.

BTW: I fixed a bug in the domstats path recently where the "last error"
was lost (see commit e1fc7ec08). Although I don't think that's what
you're chasing here.

John
Post by Nikolay Shirokovskiy
+}
+
+
static int
qemuProcessPrepareHostStorage(virQEMUDriverPtr driver,
virDomainObjPtr vm,
diff --git a/src/qemu/qemu_process.h b/src/qemu/qemu_process.h
index 2037467..52d396d 100644
--- a/src/qemu/qemu_process.h
+++ b/src/qemu/qemu_process.h
@@ -214,4 +214,6 @@ int qemuProcessStartManagedPRDaemon(virDomainObjPtr vm);
void qemuProcessKillManagedPRDaemon(virDomainObjPtr vm);
+bool qemuProcessMissingLocalOptionalDisk(virDomainDiskDefPtr disk);
+
#endif /* __QEMU_PROCESS_H__ */
Loading...