Kusama’s Governance Thwarts Would-be Attacker!

Published in

Kusama_Network

6 min readOct 19, 2021

The Kusama network, the experimental hotbed of unaudited, pre-production code serving as Polkadot’s wild cousin and somewhat rash trailblazer lived up to its “Expect Chaos” mantra and had another adventure. Fulfilling its purpose as a “value-bearing testnet”, Kusama incentivized security analysts to find and demonstrate an exploit before the code went into production usage. One KSM was incidentally left in the attacker’s account which may as well be taken as a finders fee (perhaps it would have been more had the bug in question not already been fixed in the codebase for several weeks prior to the attack).

What Happened?

The Karura team (one of Kusama’s parachains) discovered that the balance of KSM tokens held by Karura parachain on the Kusama Relay-chain did not match the total number of KSM tokens recorded in the Karura tokens pallet. To be precise, the Karura parachain account was holding 11,000.00 KSM less than could be explained. The team identified a number of suspicious XCM transactions that transferred KSM tokens out of Karura parachain account:

They then took three actions. They notified centralised ecosystem infrastructure providers about the account containing the stolen funds; they helped Karura governance to invoke a special contingency-plan pallet to disable the ability to send raw XCM transactions immediately; and they requested the Kusama Relay chain’s decentralized governance bodies to help recover the funds.

As a result of the latter, motion 373 was proposed by the council to recover the transferred funds (accounting for 10,000 KSM), followed by a Technical Committee proposal for fast-track referendum and the referendum itself. The period of the referendum and enactment was intentionally kept minimal by Kusama governance in order to maximise the chances of the recovery of stolen funds, which wouldn’t work if the attacker were to have transferred the funds during the vote:

The force transfer referendum was approved & executed, and 9,999.00 KSM were transferred back to the Karura parachain account. Note: the motion failed to execute twice prior (referenda 141 and 142) due to incorrect usage of the preimage registration extrinsics, owing to the short enactment period.

The remaining 1,000 KSM was burned by the attacker and we expect a proposal to be made to the Kusama council in due course to re-mint this missing KSM back into the Karura sovereign account.

After the nature of the issue became clear, all parachain teams were notified with clear instructions for remedial steps, if needed. Statemine, though not susceptible to exactly the same exploit, was upgraded with a bug-fix immediately as a matter of caution:

All parachains connected to Kusama network have since patched and upgraded their code if needed and no other attacks, successful or otherwise, have been detected.

How Did This Happen?

This happened due to the compounding of three factors on the Karura chain:

The misconfiguration of the XCM pallet.
The usage of older, flawed, code.
The utilization of Reserve asset transfers.

The combination of these factors led to the possibility of the attacker being able to execute a message which could make a withdrawal from Karura’s reserve account on the Relay-chain.

The unexpected misconfiguration of the XCM pallet.

While XCM v2, which is in use on the Kusama Relay-chain and is nearing audit completion without any major issues found, has some expectation of security, XCM v0 and v1 were early revisions which were not audited and as such not intended for production use. Though Kusama is an unaudited and pre-production network, owing to the large attack surface of XCM, the ability for users to execute hand-crafted XCM messages was intentionally disabled.

This can be seen in the Kusama runtime code:

impl pallet_xcm::Config for Runtime {
    // ...but they must match our filter, which rejects all.
    type XcmExecuteFilter = Nothing;
}

And in the original Statemine code:

/// No local origins on this chain are allowed to dispatch XCM
/// sends/executions.
pub type LocalOriginToLocation = ();

Unfortunately, by the time that the corresponding piece of code had come to be introduced into Karura, the comment remained unchanged but the underlying code was now something rather different:

/// No local origins on this chain are allowed to dispatch XCM
/// sends/executions.
pub type LocalOriginToLocation = (
    SignedToAccountId32<Origin, AccountId, RelayNetwork>,
);

This removed the guard rail and allowed anyone (including our attacker) to execute any XCM message on Karura, opening the door to the large attack surface of XCM v0/v1.

The usage of older code.

Kusama was upgraded to XCM v2, the first XCM version that is expected to make it through audit and on to the Polkadot Relay-chain, though Karura (and Statemine) had not yet been upgraded to XCM v2. XCM v1, it turns out, contained an issue allowing for a kind of limited privilege escalation. With XCM v2, much of this code was rewritten to be cleaner and in doing so could not harbour such a flaw.

The attacker made use of the misconfiguration in order to access the exploit in the old version of XCM. They submitted a transaction to execute an XCM message containing 𝙸𝚗𝚒𝚝𝚒𝚊𝚝𝚎𝚁𝚎𝚜𝚎𝚛𝚟𝚎𝚆𝚒𝚝𝚑𝚍𝚛𝚊𝚠, which then sent a 𝚆𝚒𝚝𝚑𝚍𝚛𝚊𝚠𝙰𝚜𝚜𝚎𝚝 message to the Relay-chain transferring Karura’s KSM into an account they control. In XCM v2, this would (correctly) fail, however previous versions of XCM contained a bug allowing it to succeed.

The attacker could not attack Statemine using the same exploit since it, being a common-good parachain, has no KSM reserve account on the Relay-chain. Unperturbed, they attempted some other attacks around a day later (approx 1600 CET 13th October), all without success primarily because the Kusama Relay-chain had already been upgraded to XCM v2 by this time.

The lessons

When introducing unaudited code, Always Verify Correctness. This is really just common sense in an industry or movement that prides itself on reducing the need for trust and authorities. No code author or company is good enough to be blindly trusted. Linus’s law, “given enough eyeballs, all bugs are shallow”, doesn’t work if nobody studies the code except the original authors themselves. Third-party professional audits help give some confidence that the code is good, as does time in the wild. But when you’re introducing code into your own codebase, it’s best to have a good understanding of what it does and how it works.
Introduce more stringent review requirements for highly sensitive code. A basic code review process is clearly not enough to avoid code changes that remove guard rails erroneously. Parity will introduce the idea of locked files and locked lines which are small pieces of highly sensitive code which require enhanced review by multiple experts in order to be changed in our repository. The guardrail which was incorrectly removed would be just such a locked line. We will of course be making this CI software available for all teams to use.
Introduce contingency mechanisms. One of the reasons that Karura was quickly able to resolve this issue was due to it containing a special pallet called “Transaction Pause”. This enabled the Karura governance body to halt most types of user transactions including the misconfigured one which enabled the attack. We would propose a similar “safe-mode” feature in Substrate chains in general, making such a contingency mechanism available for all parachains to use. In the case of a possible attack, this will be the first line of defence and will buy time for the governance process to determine the nature of the exploit (if any) and possible remedial effects including the repatriation of funds and upgrading.
Use canary networks. If exploits are going to happen, then better that they happen on a network whose entire raison d’etre is to be attacked and have vulnerabilities uncovered in the process.

FINAL NOTE:

Any non-trivial code on Polkadot must first go through third-party security audits, generally completed by the folks at SR Labs. Since XCM is not yet fully audited, NONE OF THE CODE INVOLVED HERE IS, OR EVER WAS, ON POLKADOT.