Jess Asato’s Fight for AI Accountability

Image - House of Commons

Politicians’ discussion of artificial intelligence has become commonplace within parliament in recent months, focusing on how AI can assist defence, healthcare and education. However, when Jess Asato spoke out in the Commons in January, it was to pose a very different question: how can AI software be allowed to cause harm, yet developers suffer no consequences?

At the beginning of the year, Jess Asato was the victim of AI generated explicit images of her that were shared on X, picturing the MP for Lowestoft in a bikini. X’s AI tool Grok developed the images following prompts from users. She told The Financial Times that a video ‘showing her being chloroformed and prepared for a sexual assault’ was also produced by Grok.

Asato has moved to take legal action against Elon Musk’s xAI, holding owners and developers accountable for what their platforms produce and the harms their tools enable. A legal case has been filed with the High Court, with the MP hoping her case will set a precedent which forces companies to take responsibility.

The case arrives at a pivotal time in digital development. Also in January, the UK government threatened legal action against X for its AI’s production of explicit content that was based on real underage girls and women. Deepfake content has increasingly been used to target politicians, journalists and private individuals alike, exposing how existing legal frameworks have struggled to keep pace with the rapid development of generative AI.

Prime Minister Keir Starmer backed Asato’s decision to pursue the case, stating that she is ‘absolutely right in the action she is taking’ against the ‘disgusting images’ that were created by Grok.

It was a pleasure to have the opportunity to interview Jess Asato last week, discussing her own experiences as a victim of deepfake AI, the impact of misogynistic content on women and democracy and her hopes for changes in the future.

Targeted for speaking out

By speaking out publicly about Grok’s production of explicit content, Asato herself became a victim. Her experience demonstrates a wider pattern of abuse: women who speak out often become targets.

After returning from the Christmas break from parliament, Asato spoke about the sexually explicit deepfake images Grok was producing:

‘I said that Elon Musk had the tools and the ability to stop it right here right now. There are AI image generators which do not allow you to upload photos without consent, do not seek instructions, and are not programmed to allow nudity. I'm sure for very clever people there are work grounds, but most of the time, it can't be done. Grok was created specifically to create this content. xAI wanted this to be an end project knowing that in the main, this would harm women and girls’.

Her criticism prompted further abuse:

‘When I spoke out and said Elon should stop this right here right now, I became a victim of the trend. I was put into bikini pictures and when I did a BBC interview about how this is exactly the kind of misogyny women shouldn’t have to face online, lots of people then put me in burkhas saying do you prefer that? They also made me pregnant with quintuplets and put me in a kitchen saying this is where you deserve to be. The final thing that was really quite disturbing was a video that was created after Elon Musk had retweeted a picture of me with a comment from another user. Someone created a video of me being chloroformed, by a group of men, fainting and then my skirt being lifted up by them as if prepared for sexual assault.’

Deepfakes therefore represent more than a technological problem. They are becoming another tool through which women in public life can be intimidated, humiliated and, ultimately, discouraged from participating in democratic debate.

When Asato attempted to have the content removed, she encountered another defining feature of today's online environment - moderation carried out by automated systems rather than human decision-makers:

‘When I was trying to get these bikini images and this particular awful video taken down, all my requests were rejected by X. You’re shouting at an algorithm that's looked at your complaints and just comes back and says this has not violated our terms and conditions and there’s nothing left then other than to accept it. A friend of mine suggested I might want to get a lawyer to send them a legal letter and my lawyer then said we think you might have a case against xAI for misuse of your data and privacy violations.’

This raises the question of who should decide what constitutes harm online? Increasingly, these decisions are made not by courts or elected representatives, but by private companies applying their own terms of service through proprietary algorithms. This often means an automated system will simply dismiss complaints raised by users. The case is therefore no longer simply about the removal of individual images. It is about whether technology companies can continue to position themselves as passive platforms when their products actively generate harmful content and how the law deals with this.

Aiming for digital rights

Asato hopes to set both a societal expectation and legal precedent that reshapes how technology companies are held accountable for the products they develop. At the heart of the campaign is the aim to ban nudifying apps entirely:

‘There is no public interest for these kinds of apps and the content has become rife, including within schools. If you can see a harm, you need to try and stop it. This is about trying to set a precedent that will send a signal to tech companies that they can not dismiss our rights to privacy and dignity and to be free from harm when we engage as women online.’

This represents a departure from traditional methods of policing online content. For much of the social media era, regulators focused on removing illegal or harmful content after it appeared online. Generative AI challenges that model because the technology itself can produce abusive material. In light of this, campaigners are aiming to ensure that tech companies prevent their tools from being able to create such content from the outset.

For Asato, it is clear this case is not just individual, but part of a wider picture:

‘I’m doing this on behalf of all the millions of victims who’ve had misogynistic image manipulation happen to them, but have never had an avenue to make tech companies abide by the UK law.’

Her comments highlight another defining feature of modern digital politics: the imbalance of power between individual users and multinational technology companies. Platforms such as X have become central spaces for political debate, journalism, and civic participation, yet users have relatively little influence over how these spaces are governed. In her efforts as a parliamentarian, Asato gives a voice to the many millions of victims who feel they are just one voice shouting into the billionaire (and in this case trillionaire) void, unable to make much change.

The Labour government has sought to position itself as taking AI abuse seriously and already have begun introducing measures that will hold platforms to account.

‘The Secretary of State Liz Kendall made it very clear that this was not acceptable, and I'm really pleased that as part of the pressure we were able to bring to bear at that time she announced the nudification app ban which is now in law. That came through the Crime and Policing Act and they also brought AI tools under the remit of Ofcom so Ofcom has not been able to investigate xAI because they didn’t have the tools they will do in the future, which is great.’

For years, governments relied heavily on technology companies to regulate themselves, with platforms developing their own community standards and moderation policies. However, when platforms such as X, Facebook, and Instagram are heavily influential within the political sphere as a space for debate and promotion, it brings into question whether self-regulation is enough. Expanding Ofcom's regulatory powers represents an attempt to shift responsibility away from corporate self-governance towards democratic oversight.

Campaigns and consent

Alongside her legal action Asato launched the campaign “Stop AI Abuse,” which has received a lot of support, with many signing and sharing the petition. Asato told me she’d also received many personal, heartfelt messages from people across the country ‘feeling a sense of relief that we can actually do something and we can actually stand up to tech giants who have billions, or even in Musk’s case trillions of dollars of wealth and to say it doesn't matter how big you are or how wealthy you are, we have rights too.’

Asato’s legal case against xAI is only the beginning of her hopes to change legislation, as she aims to establish stronger legal protections over people's digital identities:

‘So my overall campaign is to bring the online space under much better parity with what we would expect in offline spaces. With that in mind, I’ve been looking into whether there might be a case for some sort of individual digital consent over photos, voice and moving pictorial representation. What I'm looking at with a number of organisations and academics is whether we can introduce consent rights. We consent all the time to cookies on browsers, we are constantly consenting in online spaces. Why couldn’t we agree to either yes I'm happy to be manipulated by AI and for other people to use my image or no I’m not happy, I do not want my profile pictures manipulated by someone else?  You’re sharing my data without consent and you shouldn’t be allowed to do that. So that, in a nutshell, is why I’m pursuing this case.’

If people increasingly live, work and participate in politics online, then rights protecting physical autonomy may need to evolve to offer comparable protections in digital spaces. Asato's proposal for digital consent would represent a significant evolution in internet governance. Rather than treating media simply as data that platforms can process, it would recognise an individual's likeness as extensions of their personal autonomy, requiring explicit consent before they are manipulated.

Are AI platforms responsible for individual misuse?

For years, social media companies have argued that they are neutral platforms rather than publishers, maintaining that responsibility rests with users who misuse their services. However, the introduction of generative AI on these platforms changes the debate, as their tools are used to create harmful content. Asato argued that the principle that developers should be working towards is ‘safety by design. We expect this from our cars - to have seatbelts and airbags and automatic breaking. There are design principles that they have to follow in order to reduce harm. We have got the same expectations from our tech developers, but they just don't bother to follow them. With xAI, it is very clear that they did not care about safety by design. In fact, they inbuilt harm because they knew it was a better business model. This means they’re making money off the back of the degradation of women and children, that's morally bankrupt.’

As Asato attested, certain principles underpin regulation in sectors such as aviation, pharmaceuticals, and vehicle manufacturing, where products are expected to meet safety standards before reaching the public. If these standards are not reached, consumers can launch legal action. She proposes that social media platforms and AI developers be held to the same standards.

With many of us living our lives increasingly online, Asato called for accountability, for people to take responsibility over the content they produce, as well as for tech companies to take serious action against those creating and sharing harmful content. There appears to have become a wide acceptance that people will use the internet and social media in a negative way and that society should just tolerate this. This is something Asato described us as having ‘sleepwalked into. It’s just “oh god, it's the internet, bad stuff happens” The idea that somehow people don’t exist as people online, we’re losing our humanity. The dignity of the individual can not be dismissed. You can't have it both ways, you can say everyone gets to live their lives online, but actually that's not real life. Either it's real life and we’re really living online, or we’re not. It’s my body and it’s my choice, that's been what women for generations have said - my body, my choice and my right to do what I want with it. Only I can give consent.’

Misogyny by design?

Although deepfake technology can target anyone, its effects are far from gender neutral. Women remain disproportionately affected by sexually explicit AI-generated abuse, while female politicians continue to experience significantly higher levels of gendered harassment than their male colleagues.

In the past few years, many of us have become more aware of online spaces that exist which are designed as ‘anti-women’ environments, with the idea of incel culture and the emergence of a wider network known as the manosphere, focused on masculinity, misogyny and opposition to feminism. The work of journalists such as Louis Theroux with his recent documentary ‘Inside the Manosphere’ have exposed a variety of streamers and self-proclaimed influencers for their discriminatory content and views.

This image-based abuse therefore feeds into a broader culture of violence a discrimination against women. Asato’s experience of being placed in bikinis, burkhas, and as pregnant is not random image generation. Each image drew upon familiar misogynistic stereotypes about women's bodies, sexuality, and social roles. AI did not invent these narratives, it accelerated and automated them, making it easy to produce such content that can be shared millions of times worldwide.

Asato argues that ‘under the banner of freedom of speech, these platforms are monetising hate against women. We know the algorithms will push discriminatory content into people's feeds, because they generate uproar and gather interest. It’s more likely that humans will watch these things because we’re engineered this way. We’re interested in conflict, death, and sex, and so anything that pushes that means we're more likely to look at it, it’s just instinct.’

Since Elon Musk's takeover of X, the platform has increasingly presented itself as a champion of free expression, with moderation policies relaxed in the name of protecting speech. Supporters argue this approach limits censorship and protects open debate. Critics, however, question whether platforms can continue to describe themselves as neutral public squares when their own AI systems are capable of generating abusive content.

Social media sites are regularly using our data to target posts for specific audiences, pushing content towards individuals based on gender and age. Asato told me how concerning it is that many studies have shown that ‘if you set up a new account on Instagram, or any other social media platform, within an hour, if you're a boy of 14, you’re being pushed to misogynistic content. If you’re a girl, you’re getting influencer content, stuff on body image, all of those societal pressures. This is one of the reasons I was a huge proponent of the social media ban the government just introduced. For children and teenagers, life is really hard anyway during that time, if you’ve got social media driving these messages again and again, it's hard not to learn from that, especially if you’re spending hours and hours on there, partly because they’re feeding you this junk and it’s addictive and it’s just this perfect storm.’

Impact on democracy and women in politics

Whilst men undeniably face abuse online, their experiences are not defined by gender as women's are. Asato recalled her own experience when she stood as a candidate in 2014; she called out a local reporter who made ‘a deeply misogynistic and victim blaming comment’ when covering a high profile rape case of the time:

‘I issued a press release asking for him to withdraw the statement or apologise, alongside the male candidate next door, Clive Lewis who is now an MP. The same press release at the same time, we tagged each other in. And I just got this tsunami of you’re too ugly for rape, you fat bitch, shut your mouth, you whore. He on the other hand got ‘well done mate for speaking out’. He saw this and sort of understood what happens when you say something about violence against women and girls as a woman. Everything comes back as gendered abuse.’

‘So misogyny in online spaces has been around for a while, it's just got worse and worse and worse and AI has just supercharged it. That ability to just take somebody and make it into the worse they could be, within seconds. It’s a tool to silence.’

While robust political disagreement and debate is an essential part of democratic life, targeted harassment aimed at intimidating or silencing women risks narrowing political participation and reducing the diversity of voices represented in public institutions.  Asato told me that ‘it is having a damaging effect on women who want to be involved in democracy because they’re looking at everything we receive and thinking I don't want that. That in itself is damaging democratic representation of women.’

Rather than focusing on policy disagreements or political decisions, attacks are far more likely to target women's appearance, sexuality, and perceived roles within society. Generative AI has intensified these existing forms of misogyny, allowing abusive imagery to be created and shared within seconds at almost no cost.

Ultimately, Asato's legal challenge is about more than one MP, one technology company, or one AI model. It is a test of whether the law can keep pace with technology and hold them to account. Asato's campaign highlights the growing intersection between AI, misogyny, and democracy, raising fundamental questions about who controls the digital public sphere and how they protect users, particularly women, from abuse online.

‍