authors: Ali, Lizard, and Kat
Recap of the pilot program:
A group of contributors came together in July 2021 and decided to get yAcademy off the ground by running a 1-month pilot fellowship program.
Work started to establish an online presence for the academy, set up coordination and communication channels, define a structure for the program, select the codebases to audit, book guest auditors, and recruit and interview fellows (previously known as interns).
Fellows were recruited through a referral process where anyone could refer a candidate or apply for the fellowship. Three candidates were spotted at Gitcoin’s KERNEL’s security track, and were proactively invited to apply.
An initial screening was done with each fellow, and then a group interview was conducted where all potential fellows chose a contract to audit. Fellows were allowed to work in groups and the majority of the candidates ended up working in the same group. After the findings of this mock audit were presented, a survey was sent out to let fellows and organizers assign points to who contributed the most (similar to a coordinape circle).
The program kicked off on Sept 15th with a short introductory call to welcome and orient participants. After that, one codebase was audited each week, for four weeks. The codebases selected based on how urgent their need of an audit is based on the projected tvl they’re expected to hold. This was done in consultation with Storm from Yearn Security.
Each week began with an overview live session on Monday hosted by the guest auditor, who walks through the code pointing out its main structure, things to look out for, pointers, and challenges for the fellows to take on during the week. The session was recorded in case some fellows miss it, they could watch it at a later point.
Discussions throughout the week took place in private and bridged Telegram and Discord channels. Along with guest auditors, members from Yearn Security and Strategist groups were present and engaged with the fellows. Bounties were occasionally announced during the week to encourage fellows to use certain testing tools such as Smock and daaptools.
At the end of the week, on Saturdays, a live review session was held. Each session started with fellows sharing their screens and presenting their findings, followed by feedback from the guest auditor. The session is then concluded with a presentation or open discussion with guest speakers, who were for week 1-4 respectively:
-
Doggie from Yearn Core spoke about security considerations around the time v2 was being designed late 2020, followed by open discussion.
-
Anton from ChainSecurity: From the perspective a traditional auditing firm spoke about the process of auditing, how his team works together, how they select teams to audit, and how he got started as an auditor.
-
samczsun: approach to auditing, solo vs pair auditing, a walk through of the first inspection of a codebase, open discussion at the end.
-
Andre Cronje (who also hosted the closing ceremonies and the handing out of awards for outstanding fellows): open discussion and AMA, with participation from Doggie as well.
A Coordinape circle was set up for fellows to track the contributions, one epoch each week where fellows and organizers all provided 100 GIVE tokens to the fellows who had added the most value that week. In the final review session, the total amount of GIVE for each fellow was summed up and the top 5 fellows received an award for their contributions.
A survey was sent out to nominate people for specific rewards and provide qualitative feedback. These awards were delivered as POAP NFT’s and everyone who participated in the program received a participation POAP. The awards that were given can be seen here.
Audits:
Audits reports can be found here: https://github.com/yacademy/audits
Retrospective:
At the end of the program, feedback was collected from everyone involved: organizers, fellows, supporting devs, guest auditors, and guest speakers. The remainder of this post provides highlights from this feedback, as a retrospective for the wider community to get a closer look at how it went and the ways it can be improved as well. The raw feedback responses can be found here: https://docs.google.com/spreadsheets/d/1z5i4Aiic9q2_H5jPqv7wTFKAKP_AdY6N-UkFPTg3SqQ/edit?usp=sharing
Overall assessment of the program:
Overall, everyone enjoyed the session, especially the contract developers. All 4 contract developers gave the highest rating possible.
What went well:
-
Found bugs in prod or pre-prod code
“Results were outstanding. We hired a security firm to audit the exact same version of the smart contract and they missed most of the issues reported by fellows. Clarity of communication / report presentation was superb.” - Contract Developer
-
Organization/structure of the program
“The organization of the course was perfect” - Fellow
-
Small group auditing and networking
“I liked working with groups, and the guest speakers brought a lot to the table. Was very cool to be able to learn from all of the fellows during all of our live calls.” - Fellow
-
Guest Speakers
“I learned a lot from guest auditors explaining their process” - Fellow
Areas of improvement:
-
There is no path for fellows to get hired as auditors
- “The program was fun, but there is long way to go in training auditors” - Organizer
-
More time for presentations
“The presentations fellows gave were a bit rushed and hard to follow, would have also liked to communicate with the more as I think a lightbox approach to auditing is usually beneficial.” - Contract Developer
-
More organization and communication
“Personally I was a bit lost in the call organization. There was a shared calendar but there were usually questions about schedule right before each call, so maybe overall schedule organization could be done simpler.” - Contract Developer
-
More structure around process and tools for Audits
“I felt a little confused about what was expected of me during the beginning, as someone with experience writing contracts but zero experience on the auditing front. It was just sorta “get in there and figure it out” By the end of the program, I was much more comfortable re: specifically what I’m looking for in the contracts, but a “week 0” of sort might have helped me establish a workflow before auditing my first contract.” - Fellow
-
Fellow Engagement
“I should have been more active, so I guess thats on me.” - Fellow
How much did it cost?
yAcademy has spent 55k USD since its inception:
- 15k on Gitcoin KERNEL sponsorship
- 40k on rewards for the fellows of the pilot program.
Since YIP-53 doesn’t specify any spending on contributors rewards, the oversight committee decided not to grant any funding for contributors.
Someone from the community provided 15k to cover contributor rewards for the pilot program:
- 11.5k for ops/comms contributors
- 2.5k for Web presence admin costs and auditor job ads
- 1k for NFT artist.
Top 5 Fellows:
-
Anon - The Architect
“I was familiar with debugging in general before joining. I was interested in learning more about some of the vulnerability and attacks that are more specific to smart contracts, such as reentrancy, calling the constructor twice, flash loans, etc. I also was interested in the auditing process. How do you more formally approach and review a contract? How do you ensure the review is thorough enough that you can sign off on the contract as audited?” - Anon- “[Anon] came up with an orderly process to review the code and submit the reports and basically carried the first week on his own.”
-
Nibbler Express - The Sentinel
- My experience is more on hardware and cryptography, and had been doing Solidity for about 6 months.
- “Nibbler seems to have a knack for spotting issues quickly”
-
Al Dent Spaget - The Diplomat
I wanted to get a better feel for performing more official audits and get some hands on experience analyzing the latest vulnerabilities in DeFi. The auditing practice and protocol exposure were great takeaways, along with working with others and learning from some of the top security experts and devs in the space. - Al Dent Spaget on why he joined yAcademy
- Been working as a dev for about half a year
- “Al Dent Spaget was quick with a thank you or a kind word for everyone involved.”
-
Justin Bebis - The Wizard:
It was appealing because lots of knowledge in defi is siloed within individual companies so any opportunity to break in and gain some is very valuable. I am always trying to become a better engineer and I couldn’t pass up the chance to learn from and collaborate with those who feel the same. - Justin on why he joined yAcademy
- Full-time solidity developer for a couple of years.
- “JB is familiar with best practices and thoughtfully analyzed how contracts compared to these practices.”
-
Dhruv - The Champion
I was motivated to learn the skillset of auditing strategy contracts from multiple aspects (both the manual reading of contracts and writing fuzz tests, symbolic testing of the possible state changes violating specs etc) without being excessively relying on the automated tools. Also, another major point was to collab with devs for doing the audits of the complex strats and mathematical bugs, and meeting the prominent devs in the ecosystem and sharing their experiences , it’s must on getting the learning curve to be a professional auditor in quick time - Dhruv, my motivation
- 6 months experience in solidity , worked on integrating defi AMM’s ( kyber , uniswap and makerDAO ) into the financial protocols for companies like opendefi , RequestNetwork.
- “Dhruv was enthusiastic and was not afraid to ask questions.”
What is next:
The most frequent question yAcademy contributors have been receiving is a variation of: when does the next program start, how do I enroll, or how do we get yAcadmey to audit our codebase?
In an upcoming post, members of the yAcademy community will share a post outlining the path forward, and the various options to make the academy sustainable, and the plan for growth.
In the meantime, any and all feedback from the wider community on what has been reported in this post is appreciated. Contributors are also always welcome to help in their areas of competence. If you would like to get involved, please contact team@yacademy.dev
Got bugs? Peace out.
Appendix
Raw feedback responses by fellows, organizers, devs and guest auditors
