IPFS Gateway Conformance Testing

Goals of the project

The main goal was to build a testing framework and a conformance test suite for the IPFS HTTP Gateway specification that can be easily maintained by any engineer working on the stack. The test suite needed to be approachable to engineers of various levels and familiar with different programming languages. It should also be easy for any new project to use this test suite in their own CI, with minimal cost and effort.

We also shared a detailed Case Study for this project

Rough Timeline

February 2023	First JavaScript proof of concept - Data-Driven Approach Validated with the team
March 2023	Second Go proof of concept - Stack & Syntax Validated with the team
April 2023	Subdomain Testing, JUnit Report generation Onboarding of three HTTP gateway implementations: Boxo, Kubo, and Bifrost-Gateway
May 2023	v0.1 - DNSLink Testing, Dedicated String Templating
June 2023	v0.2 - Custom Fixtures Support (CAR files), CORS Testing
July 2023	v0.3 - Kubo’s Test suite completely ported to Gateway Conformance

IPDX Special Expertise Benefits

IPDX brought our domain expertise to design a data-driven test suite, making it easy to refactor, transform, translate and rethink the test suite. We relied on native Go testing and reused our ecosystem of tooling to provide reusable workflows and other IPDX quality enhancements. Our approach enabled functions to transform and generate additional tests as needed, significantly reducing maintainer workload.

The Results

The Gateway Conformance test suite is now a part of the IPFS Community's toolbox. We added the test suite to three core projects, removed thousands of lines of shell-based tests, and produced reports with thousands of checks which caught significant issues. The project is live and open source on Github.

The Deliverables

A user-friendly, data-driven test framework for the IPFS HTTP Gateway specification
Porting over a thousand test cases from shell scripts
Implementing essential features like subdomain testing, fixtures support, etc.
Distributing the test suite as a reusable GitHub workflow

Links and references

Client Feedback

📎

Collaborating with the IPDX team saved our core team months of work, allowing us to focus on the protocol where our expertise truly lies! The IPDX team was committed to understanding our needs, conducting thorough research, and producing high-quality work. Their methodology alleviated much of the burden on our end. Throughout the process, we had the chance to assess the ongoing changes, and the outcome has exceeded our expectations. And not only is my team bettered, this is a significant asset for the entire IPFS implementer community, paying dividends across various implementations running in different languages. I could not be happier with the collaboration, and would recommend their quality services in a heartbeat!

Steve Loeppky, Engineering Manager PL IPFS Engineering Team