AMP 3 AI usage disclosure
AMP 3 is the first major version of AMP that has been built since the advent of modern AI. Development on AMP 3 originally started in 2023 before a lot of the modern AI tooling existed, but we have chosen to use it responsibly in areas where it makes sense to and is well justified.
The overall reasons for AI usage broadly fit into four categories:
- It was in our users interest because it allowed us to do something for their benefit that would otherwise be challenging.
- It was better than using other alternatives to achieve the same goal.
- It saved time and effort where the difference was neutral, that is a human would have produced identical or very similar output.
- It had no effect on what was actually shipped to end users.
Below we have outlined the areas for AMP 3 where AI has been used, and what the justification for that use was.
Localisation
AI was used to produce all non-English locales (including STE100) by translating human-written English text.
Justification
There are a few core justifications for using AI for this particular task:
-
During development, it was useful to have localised strings already available so we could evaluate how they would change the layout and flow of text on screen, and maintain accessibility requirements and general visual polish in situations where the text being longer or shorter in one langauge than another would change the page layout.
-
It means that users can have a usable localisation at launch time. Any issues with poor voicing or inaccurate translations can be addressed post launch once it’s in the hands of native speakers.
-
It’s based on a human-authored ground truth as to the intent and message being expressed.
-
It produces higher quality translations than existing non-AI based alternatives.
Accessibiltiy and compliance.
AI was used as a first-pass to make sure that accessibility features were properly implemented (such as role and aria attributes). Subsequent passes were done via manual testing.
It was also used to perform automated testing of accessibilty standards with regard to details such as text size, contrast ratios, hit box sizes, etc.
Justification
A lot of the accessibility tooling is very consistent, taking the form of adding attributes, descriptions and hints for screen readers and other accessibility tools. AI made it possible to do the initial bulk pass very quickly and in a way that wasn’t materially different from how a human author would approach it since it involves following a set of well defined standards that would always be implemented the same way.
AI was also to itterate on these details. So after making a set of changes, an AI could then load the page up in a browser and check that these features met the documented standards automatically which helped improve the scope of an otherwise neglected area of software development.
Because the changes required were individually very simple, but large in number - we felt that using AI to help improve the lives of users with accessibility needs was well justified and in our users best interests.
Bulk unit test authoring
Unit tests are a type of automated test that are run automatically at compile time. They test individual features of the softwares functionality in isolation. One of the main purposes of unit tests is to catch regressions, that is where a change in one system breaks or changes the behaviour of another system compared to when it was originally written.
AI was used to automatically build many of the unit tests as the original functionality was being written.
Justification
Unit tests tend to take a very straightforward pattern:
- Perform a task
- Check that the task happened as expected
- Check that any other output matches an expected output
- Unexpected errors did not occur
As code for unit tests is so consistent and procedural, AI written tests are not significantly different from those authored by humans, but using AI to automated this task means many more of these tests can be written in a shorter period of time, and gaps that humans are likely to overlook are less likely to occur because the tests are generated to cover the entirety of the code that was written.
Because code written for unit tests are not part of what is actually shipped to users (these do not run on your systems) and help us to maintain the quality of the software, we went this was a well justified use of AI.
Disposable ‘Backend Excercising’ interfaces.
As a backend feature is built, one of the challenges involved with testing it is actually running it in-situ on a real system because this requires building out a user interface to actually leverage the new system. The problem with this is that you often end up prematurely building a frontend interface that doesn’t ultimately make sense because it was focused around excersing the backend rather than what is actually best for the user.
During development, we used AI to build disposable frontends. These were basic, unfriendly interfaces with no styling or refinement whose job was purely to help test backend features properly.
These were then removed and seperate human-designed interfaces were put in to replace them with a focus on what is actually user friendly and practical to use rather than just “Use the backend”.
Justification
Because these interfaces were disposed of and never used (they were ugly, unfriendly, unintuitive and not something you’d want to use) and existed purely as a technical tool, the benefit gained in helping to speed up testing and development of backend features we felt was well justified, especially since like the unit tests this was code purely used internally and never shipped to users.
Migratory Code
While AMP 3 is a new platform compared to AMP 2, it shares a certain amount of logic with AMP 2’s generic module since it consumes the same templates. AI was used to help migrate existing human-authored code from the old platform to the new one.
Justification
Because the original code was already human written in the first place and in the same programming language, AI was merely performing the role of migrating that existing code as-is into the new system, making only minor changes to make it fit the new platforms design patterns. The changes were generally minor and similar in style to how a human doing these bulk changes would go about them. So the quality of the result was not adversely affected while saving time and effort.