Mempool error

Hello @Support

Is there any way to fix this quickly (i.e. without resyncing from the scratch)? None of my validators can work now because of the error below. They have entered to the endless loop giving this error.

config/mainnet
2022-11-10 10:35:37.204 incognito.go:109 [INF] Server log: Version 1.19.0-beta
2022-11-10 10:35:38.512 incognito.go:143 [ERR] Server log: could not open connection to leveldb
2022-11-10 10:35:38.513 incognito.go:144 [ERR] Server log: -2000: leveldb: manifest corrupted (field 'comparer'): missing [file=MANIFEST-000618]
levelvdb.OpenFile /data/mainnet/mempool
github.com/incognitochain/incognito-chain/databasemp/lvdb.open
        /Users/autonomous/projects/incognito-chain/databasemp/lvdb/db.go:16
github.com/incognitochain/incognito-chain/databasemp/lvdb.openDriver
        /Users/autonomous/projects/incognito-chain/databasemp/lvdb/driver.go:26
github.com/incognitochain/incognito-chain/databasemp.Open
        /Users/autonomous/projects/incognito-chain/databasemp/driver.go:31
main.mainMaster
        /Users/autonomous/projects/incognito-chain/incognito.go:141
main.main
        /Users/autonomous/projects/incognito-chain/incognito.go:288
runtime.main
        /usr/local/go/src/runtime/proc.go:255
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1581
Open database error
github.com/incognitochain/incognito-chain/databasemp.NewDatabaseMempoolError
        /Users/autonomous/projects/incognito-chain/databasemp/error.go:61
github.com/incognitochain/incognito-chain/databasemp/lvdb.open
        /Users/autonomous/projects/incognito-chain/databasemp/lvdb/db.go:16
github.com/incognitochain/incognito-chain/databasemp/lvdb.openDriver
        /Users/autonomous/projects/incognito-chain/databasemp/lvdb/driver.go:26
github.com/incognitochain/incognito-chain/databasemp.Open
        /Users/autonomous/projects/incognito-chain/databasemp/driver.go:31
main.mainMaster
        /Users/autonomous/projects/incognito-chain/incognito.go:141
main.main
        /Users/autonomous/projects/incognito-chain/incognito.go:288
runtime.main
        /usr/local/go/src/runtime/proc.go:255
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1581
2022-11-10 10:35:38.513 panic.go:1047 [WRN] Server log: Shutdown complete

Yikes, database corrupted issue!!! I guess the only way to get rid of the issue is to clear the database and re-sync. But you can use the bootstrapping data to speed the process up and @Jared can help.

1 Like

Too bad :frowning: Contabo again throttled my SSD and I may not have enough time for even bootstrapping since some of my nodes are so close to the committee. Thanks.

No need to clear the whole data folder. Since only mempool data is corrupted, you can just remove the mempool folder only. Remember to stop the node beforehand.

Too late. I think I tried deleting the content of mempool folder but not itself. Not sure, I tried many things in hurry :slight_smile: Anyway, I’ve moved all of my nodes from Contabo. I would do it next week but I had to do this week. If I had had 1 hour more, even my migrated nodes wouldn’t have been slashed. I could have caught the committee by %50 vote ratio :slight_smile: