← All stories

Reported The Expert It Isn't Dangerous

New York City's official business chatbot told owners they could break the law

A city-run assistant for small businesses advised taking a cut of staff tips and refusing housing voucher holders. Both are illegal in New York.

Watch for

Fluency is not qualification, and an official logo is not a legal opinion.

What it said
Yes, you can take a cut of your worker's tips.

New York City launched MyCity in October 2023 to help small business owners find their way through city regulation. It was built on Microsoft's Azure AI services and trained on the city's own web pages.

In March 2024 The Markup tested it and found it repeatedly gave advice that was illegal under New York law: that an employer could take a share of workers' tips, that a landlord could turn away tenants with housing vouchers, and that there was no limit on residential rent.

The advice carried the city's name, which is what made it dangerous. A business owner following it risked a discrimination claim or the loss of a licence.

Has an AI ever done something you couldn't quite believe?

Good or bad. The disaster, the near-miss, the one that made you laugh, the one that turned out better than you'd any right to expect.

Tell us what happened

A person reads every submission. We publish your first name and your city — never your email or anything else. You can take it down whenever you like.