I struggled. I questioned myself. I almost quit.

All before I found a new approach to reviewing code. And a bug that could make Lido oracles crash.

Part 1: I'm an auditor

Lido v2 is tough. So many contracts, countless lines of Solidity code, roles everywhere, unstructured storage, oracles. You name it, it's there.

But I'm no stranger to such monsters. So I was eager to face this one. My plan: do what I do best. What I would do in my old days as an auditor.

Drawing from my past experiences, I began the security spotcheck reading each line of code. Meticulously. Trying to understand each function, from simpler to more complex ones. That would be my way to fully understand Lido and uncover a vulnerability.

Day 1: OK.

Day 2: OK?

Day 3: really, OK?

Day 4: hmm.

Day 5: not OK.

What was going on?

I was distracted. Puzzled. I couldn't craft, let alone follow, non-trivial trains of thought. Seldom did I struggle that much. Something was different this time. I was trudging through the code. My auditor weapons felt heavy and rusty. Meanwhile, the monster only grew more fearsome.

"I'm quite lost in the spotcheck, Lido is huge. Last week I approached it as a an audit, line by line. It was totally demotivating." – A message to my frens at The Red Guild.

I tried switching contracts. Features. Functions. Anything that would shake me out of the rut. To no avail.

It wasn't long until the questioning began.

Part 2: Am I an auditor ?

I was tired and frustrated. So much that I dropped the thing altogether. Stopped auditing Lido v2, to spend a few days on other tasks at the guild. Hoping that the distance would favor me with a fresher perspective on the problem.

Why was the job feeling so grueling, so unstimulating? Was the auditor mindset helping me? Could I shift it?

Could I go from a rather passive reading approach to something more creative?

Would an auditor do that?

I always tell people that a security spotcheck is not a smart contract audit. Why, then, was I thinking and acting as an auditor?

Such questioning brought me to a realization. The simplest of the ideas crashed in my mind. If I'm not having a good time acting as an auditor, and this is not an audit...

I would stop being an auditor.

Part 3: I'm not an auditor

Sounds silly, doesn't it? It wasn't for me. It's not easy to let go of the costumes we're so used to wearing.

Now that I was in the questioning mood, I thought of using it to my advantage. The next day I was back at the code, with a new plan.

I would stop passively consuming lines of code. No more line-by-line reading. Instead, I would follow a dead-simple, 3-steps framework to review the code:

1) Ask a question.

2) Find the answer in the code.

3) Repeat.

"As of today, I'll try a new approach, more based on answering questions that I come up with, and documenting them" – Updating my Red Guild frens of the situation.

I'd avoid questions that were too blurry and general. Otherwise I'd get too lost wandering for an answer. The more specific the question, the easier it'd be to stay laser-focused on it. I didn't care if my questions were too basic or complex. They were triggers. Gateways to exploring Lido in novel ways.

"Not sure if this is the best way, but I've been trying it for a couple of hours and it seems more agile and fun." – Me sharing first impressions.

Intuition, curiosity and experience helped. I gradually started building a collection of questions and answers about specific parts of Lido. From the most general, to the more specific ones.

Look at these random samples:

As days passed, I felt real progress. My Q&A set was growing organically.

Each question helped shape the next ones, creating a trail of my review. Newer questions felt more mature than the early ones, and I felt more confident to dive deeper and deeper into the codebase.

I even went public sometimes.

Me publicly bothering Lido devs with nothing but a 'somewhat specific' question. Source.

I wasn't constrained by anything but my curiosity. Which soon led me out of the smart contracts. I began exploring, and understanding, withdrawals and exits of validators.

Here's me asking my friends about validators exiting:

"Somewhat unrelated question about PoS validators: when they exit, do they go straight from 'active' to 'exited'? Or is there an intermediate state like 'exiting'? Not sure what the transition looks like. Is it immediate? Is there a delay? Any steps in between? I'll look for the answer in https://eth2book.info" – Asking frens about validators exiting.

That led me into a day-long rabbit hole into reading and comparing consensus nodes code to check whether they handled exited validators differently. Unrelated to Lido really, but hey, it was worthwhile.

Back to Lido, soon I was out of the smart contracts, looking at the oracles' off-chain code. Of course, asking more questions.

I had seen lots of potential reverts in the Solidity code. I wanted to learn how these errors were handled in the oracle's Python code.

There was a call that got me thinking. It used a try/catch to handle some errors, but I wondered whether that was enough.

I began listing each possible exception I could identify that must be handled. That led me into reading the source code of web3.py, the Python library used by the oracle.

The oracle was calling the function wait_for_transaction_receipt. Which is defined here. That can potentially raise the TimeExhausted exception.

Was TimeExhausted handled by the oracle? Nope.

So that's a bug! After considering multiple scenarios, we decided it wasn't severe enough to require a private disclosure.

The unhandled exception would cause the oracle to crash. But only after 120 seconds of asking many times for the receipt without receiving it back. Not that unlikely, although didn't seem risky for Lido.

Therefore, we just submitted a public issue for the developers.

Unhandled exception in oracle transaction submission · Issue #389 · lidofinance/lido-oracle
Hey folks, I think there’s a case of an unhandled exception that might lead to unexpectedly halting the oracle service. It doesn’t look worrisome, so I’m reporting it publicly here for you to take…
GitHublidofinance

Is it big? I wouldn't say so.

But after so much questioning, it was rewarding.