Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: TestLossOfQuorumRecovery failed #124387

Open
cockroach-teamcity opened this issue May 18, 2024 · 16 comments
Open

cli: TestLossOfQuorumRecovery failed #124387

cockroach-teamcity opened this issue May 18, 2024 · 16 comments
Labels
branch-master Failures on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv KV Team X-duplicate Closed as a duplicate of another issue.
Projects

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented May 18, 2024

cli.TestLossOfQuorumRecovery failed on master @ 93ad913106b6f0f6ec98bc2cfa788ff6d8085bd4:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery1886372325
    test_log_scope.go:81: use -show-logs to present logs inline
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json]
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery2740169656/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery1886372325
--- FAIL: TestLossOfQuorumRecovery (53.26s)

Parameters:

  • attempt=1
  • race=true
  • run=3
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/kv @cockroachdb/server

This test on roachdash | Improve this report!

Jira issue: CRDB-38862

@cockroach-teamcity cockroach-teamcity added branch-master Failures on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv KV Team labels May 18, 2024
@cockroach-teamcity cockroach-teamcity added this to roachtest/unit test backlog in KV May 18, 2024
@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 93ad913106b6f0f6ec98bc2cfa788ff6d8085bd4:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery490737238
    test_log_scope.go:81: use -show-logs to present logs inline
debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json
[debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json]
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json\": open /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1020529722/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery490737238
--- FAIL: TestLossOfQuorumRecovery (43.28s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ e4d744226f47e37b9179632711067277b2b0ab80:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery2006475443
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery2006475443
--- FAIL: TestLossOfQuorumRecovery (56.42s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@kvoli
Copy link
Collaborator

kvoli commented May 20, 2024

The test is getting tripped up finding the node-1.json file.

/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery2206972140/node-1.json: no such file or directory

This reproduces on the first attempt with --race.

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ d146ecff6f687e438706cf63591cafca60cc116d:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery1679014351
    test_log_scope.go:81: use -show-logs to present logs inline
debug recover collect-info --store=/var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/store-1 /var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json
[debug recover collect-info --store=/var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/store-1 /var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json]
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/recovery-plan.json /var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/recovery-plan.json /var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json\": open /var/lib/engflow/worker/work/2/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery688954486/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery1679014351
--- FAIL: TestLossOfQuorumRecovery (62.28s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 74d5e3e8021e53d27d823ef7f8694d59df8c3414:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery2381170779
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery398467435/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery2381170779
--- FAIL: TestLossOfQuorumRecovery (32.37s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ b5081e997adaf5950c9b85dfb6655f226bf16d29:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery3497700717
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json\": open /var/lib/engflow/worker/work/1/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3913649780/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery3497700717
--- FAIL: TestLossOfQuorumRecovery (43.35s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ e999ede1ed2bc87b34e01eedba1da58e09b8c4d3:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery2040517784
    test_log_scope.go:81: use -show-logs to present logs inline
debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json
[debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json]
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json\": open /var/lib/engflow/worker/work/0/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery543497606/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery2040517784
--- FAIL: TestLossOfQuorumRecovery (52.57s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 0ea8b622c5354a891cfb867d8d872a0746ba77b3:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery2410259060
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json\": open /var/lib/engflow/worker/work/0/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2708355366/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery2410259060
--- FAIL: TestLossOfQuorumRecovery (32.12s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 46b6fab1cbc3c25229f0487711f9a5210ed6fb31:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery1387788284
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2373033329/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery1387788284
--- FAIL: TestLossOfQuorumRecovery (53.47s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ d5f825ce04362a75781b75e19cf429f2ed5ed61b:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery1084667291
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery1754731743/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery1084667291
--- FAIL: TestLossOfQuorumRecovery (55.07s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ e5f5597b1c1570fff03ee6dfd4a1e8be65c0e008:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery4072590901
    test_log_scope.go:81: use -show-logs to present logs inline
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json]
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery2276823754/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery4072590901
--- FAIL: TestLossOfQuorumRecovery (32.33s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@miraradeva
Copy link
Contributor

Duplicate of #124396.

@miraradeva miraradeva added P-1 Issues/test failures with a fix SLA of 1 month C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. X-duplicate Closed as a duplicate of another issue. labels May 30, 2024
@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 64bd4bbac8f8d27a2fe4968404cc3d048480375f:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery628676283
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery905773074/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery628676283
--- FAIL: TestLossOfQuorumRecovery (53.52s)

Parameters:

  • attempt=1
  • race=true
  • run=3
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ a21a5fc2c955415a5e1b44cf0d995e1048f667c9:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery893642250
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/store-1 /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/recovery-plan.json /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json\": open /var/lib/engflow/worker/work/3/exec/_tmp/0ede0149c79dc299cd458e6d0ea4f601/TestLossOfQuorumRecovery3051545086/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery893642250
--- FAIL: TestLossOfQuorumRecovery (42.44s)

Parameters:

  • attempt=1
  • race=true
  • run=1
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ c84a89569fab4cc4896a0234bec1e1822dd420d8:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery3780743331
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/store-1 /var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/recovery-plan.json /var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json\": open /var/lib/engflow/worker/work/1/exec/_tmp/33cad9ecca6f841bff6186de727627fd/TestLossOfQuorumRecovery3921904848/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery3780743331
--- FAIL: TestLossOfQuorumRecovery (53.15s)

Parameters:

  • attempt=1
  • race=true
  • run=2
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

cli.TestLossOfQuorumRecovery failed on master @ 3d9eaa7939e4fb5aa0518ba7e19750a55c93cf11:

=== RUN   TestLossOfQuorumRecovery
    test_log_scope.go:170: test logs captured to: outputs.zip/logTestLossOfQuorumRecovery3216461640
    test_log_scope.go:81: use -show-logs to present logs inline
[debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json]
debug recover collect-info --store=/var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/store-1 /var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json
ERROR: failed to collect replica info from local stores: key /Local/RangeID/1/r/RangeGCThreshold does not have /Local/Range prefix
[debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json]
    debug_recover_loss_of_quorum_test.go:238: 
        	Error Trace:	github.com/cockroachdb/cockroach/pkg/cli/debug_recover_loss_of_quorum_test.go:238
        	Error:      	"debug recover make-plan --confirm=y --plan=/var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/recovery-plan.json /var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json\nERROR: failed to read replica info file \"/var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json\": open /var/lib/engflow/worker/work/0/exec/_tmp/ba24ddb5b5aa6d433fda6d631887a38c/TestLossOfQuorumRecovery3077024577/node-1.json: no such file or directory\n" does not contain "- node n1"
        	Test:       	TestLossOfQuorumRecovery
        	Messages:   	planner didn't provide correct apply instructions
    panic.go:626: -- test log scope end --
test logs left over in: outputs.zip/logTestLossOfQuorumRecovery3216461640
--- FAIL: TestLossOfQuorumRecovery (42.59s)

Parameters:

  • attempt=1
  • race=true
  • run=3
  • shard=12
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures on the master branch. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-kv KV Team X-duplicate Closed as a duplicate of another issue.
Projects
KV
roachtest/unit test backlog
Development

No branches or pull requests

3 participants