Examining the Risks Posed by Artificial Intelligence Committee of the Whole · October 5, 2026 · 10hrs 8m Source: https://hearinghearings.nyc/hearings/committee-of-the-whole-examining-the-risks-posed-by-artificial-intelligence-2026-10-05/ Video: https://councilnyc.viebit.com/vod/?s=true&v=NYCC-PV-CH-CHA_261005-110828.mp4 ================================================================ (00:00:03) Aldebol. Yeah, okay. Okay. And was... let me... I'm sorry, Philip, and yeah, yes, we are... ask you here? Oh, you... I think right, well... (00:01:46) Quiet, please. Quiet, please. Good morning, ladies and gentlemen, and welcome to today's New York City Council Committee of the Whole. At this time we ask you to silence all cell phones and electronic devices to minimize disruptions throughout today's proceedings. If you have any questions, please get the attention of one of the Sergeants at Arms. If you wish to testify, please see the Sergeant at Arms outside in the rotunda. At no time throughout today's proceedings are you to approach the dais. We thank you for your kind cooperation. Speaker, we are ready to begin. (00:02:14) Good morning, everyone. We welcome everyone to the chambers. We are thrilled to see a packed hall today. Today's hearing is truly no ordinary hearing, because today we are convening the New York City Council's Committee of the Whole, which is the entire City Council. We will delve into a deep discussion about our ongoing technological revolution, a revolution that at present is igniting incredible scientific and medical innovation and breakthroughs, but simultaneously raises serious existential safety concerns. In the past few weeks we have heard extremely concerning reports regarding artificial intelligence, reports that have appeared with increasing frequency and that highlight the potentially catastrophic dangers of this technology. New Yorkers' concerns stem from the disturbing cyber attacks committed by rogue agents, but also from extremely dire statements made by leaders of the very companies who are developing this technology. The dialogue and the legislative process we are initiating today is particularly important given the lack of any regulation by the federal government, whose actions thus far have been limited to an executive order about what we should call this technology and a pact with leading AI firms focusing on self-regulation. Our hope is that this hearing will not only allow us to develop effective regulations here in our City, but that it will spur a meaningful conversation and action in other jurisdictions, especially on the federal level. Our goals in today's hearing are straightforward: to better understand the potential threats, dangers and liability posed by AI development; to identify the benefits that responsible AI can bring to New York City government and to its residents; and to discuss the slate of City Council Bills which seek to provide responsible safety guardrails for AI. These Bills include a wide range of both proactive and defensive measures, from legislation that requires artificial intelligence systems to pass a third-party validation that is free from conflicts of interest and incentivizes whistleblowers who come forward with a portion of the penalties assessed, to proposals focusing on emergency preparedness, to be able to identify system breaches and to plan how the City will respond if such a breach occurs. To achieve these goals we have requested the participation of five firms leading the research and development of AI: OpenAI and Google, Meta and SpaceX. Today four of them will be testifying under oath before the Council, not simply for New Yorkers, but for the entire world to hear. We will indeed be the first legislative body to secure testimony from these companies under oath in the wake of these latest disturbing reports. SpaceX AI, however, is not here at all, in direct violation of the subpoena that we issued last week, and we are pursuing that subpoena in court. We will also hear from a number of City agencies to receive their perspectives on these issues and on the Bills that we have proposed. We intend to work constructively with the administration to ensure that any policies being drafted, amended, adopted and enforced will work effectively to serve New Yorkers. We will also hear from whistleblowers, industry representatives, experts and academics who we have been in touch with since we announced this hearing, and of course we look forward to hearing from any member of the public who was invited to provide testimony about these topics. I want to be very clear about our intent. This hearing is not about stifling innovation. New York City is fast becoming the tech and AI capital of the world, home to nearly 400,000 jobs in that sector and some of the companies most consequentially shaping the future of artificial intelligence, and that is something we should celebrate. We welcome and appreciate how AI has accelerated scientific discovery, advanced medical treatments, boosted economic development and helped governments and businesses deliver services more effectively while reducing costs and rooting out waste and fraud. When companies are developing technology with the potential to fundamentally reshape our economy, our workforce, our public safety and our daily lives, and when leaders of that very industry are themselves warning government about its risks, the public deserves answers and solutions. So the true intent of this hearing is to ensure that innovation happens responsibly, that the government does its job to protect the public as the technology advances, and that the protections at the municipal level can potentially serve as a national model. I spent a large portion of my life being a regular attorney. The idea that artificial intelligence is going to self-regulate defies all reason. We do not ask the airline industry to self-regulate. That is why we have the... and if there is an accident, there is an independent body to investigate that. We do not ask financial firms to self-regulate. We have the SEC and other agencies as well. We do not ask the pharmaceutical companies to self-regulate either. That is of course why we have the... So we should not ask AI companies to self-regulate either. The spirit we seek to embody in the Council's approach to AI is well illustrated in the words of tech journalist Karen Hao in her book "Empire of AI." She writes: the future of AI, the shape that this technology takes, is inextricably tied to our future. The question of how to govern AI then is really a question about how to ensure we make our future better, not worse. New York has always embraced the future. The Council is going to make sure we are prepared for it. And it is now my... (00:08:43) ...pleasure to turn the microphone over to our Technology Chair, CM Carmen De La Rosa. Thank you, Speaker Menin. Good morning, everyone. I am Carmen De La Rosa, Chair of the Committee on Technology. Today I am pleased to join Speaker Menin and my colleagues as we co-chair this hearing by the Committee of the Whole examining the risks posed by artificial intelligence. I am incredibly grateful to all who took the time to join us to discuss this important topic: industry representatives, experts, elected officials, colleagues, agencies, our brothers and sisters in labor, and especially the many New Yorkers who have signed up to testify and have submitted testimony to put their voice on the public record. Today should act as a foundation for openness in a discussion that has escalated to alarm, confusion and fear, and it needs resolution. Artificial intelligence is advancing rapidly and becoming increasingly integrated into everyday life. In just a few years we have seen it evolve from algorithmic tools that analyze data, make predictions and support decision-making into generative systems that can create text, code, sounds and images. More recently, agentic AI has emerged, enabling agents to take actions on a person's behalf. AI can be a powerful tool for efficiency, innovation and productivity. At the same time it introduces significant risks, including exacerbating biases, surveillance concerns, breaches, negative public health outcomes, violence and disparity. These are concerns that become even more serious... (00:10:21) ...as agents become more capable. When we face serious concerns or challenges, the appropriate response is to address them, not to postpone them. (00:10:32) In an effort by the City Council under the leadership of Speaker Menin, today we respond to some of these concerns. We will be hearing the following: the impact of algorithmic tools on City employees and changes in employment responsibilities due to algorithmic tools; Int 0504-2026, sponsored by Deputy Speaker Williams, in relation to prohibiting unauthorized depiction of public officials by artificial intelligence; a Preconsidered introduction sponsored by Speaker Menin in relation to third-party validation and shutdown capabilities of artificial intelligence models; a Preconsidered introduction by Speaker Menin in relation to civilian enforcement and artificial intelligence violations; a Preconsidered introduction sponsored by Majority Whip Kamillah Hanks in relation to requiring reporting and public disclosure of artificial intelligence safety incidents concerning City contracts; a Preconsidered introduction sponsored by CM Maloney in relation to establishing a private cause of action for certain harms arising from third-party misuse of artificial intelligence models; a Preconsidered introduction sponsored by CM Frank Morano in relation to chatbot data privacy, security and transparency; a Preconsidered introduction by CM Chi Ossé in relation to requiring the creation of an artificial intelligence model emergency response plan; and, not finally, one more: a Preconsidered introduction sponsored by CM Kevin Riley in relation to City contractors and subcontractors posting information concerning whistleblower protections and clarifying whistleblower protections for reporting conduct related to the use and development of artificial intelligence models; and finally a Preconsidered introduction sponsored by CM Carl Wilson in relation to requiring certain disclosures and prohibiting deceptive representations in the promotion of AI. While we work to protect people from risks associated with AI, we must not overlook the digital divide in our City. Our City is full of opportunity, yet some residents still lack reliable, affordable broadband access. Some people do not yet know how to use AI tools, while others cannot even afford the internet connection needed to access them in the first place. We cannot seek to benefit from innovation while leaving some New Yorkers behind or at its mercy. Technology being used by New Yorkers should be built for them. This is an opportunity to also explore standing up infrastructure so that our communities can become literate. I look forward to hearing from the industry and experts about this technology and how it is impacting our City, what safeguards exist and where solutions are possible as we consider how to govern AI. I also look forward to working with our state and federal partners to ensure that New Yorkers fully benefit from the advancement of AI without compromising our safety and privacy. (00:13:42) I would like to take a moment to thank the Technology Committee staff and all of the staff in the City Council, but most especially our legislative counsel... and our policy analyst Eric Brown, as well as my team, Chief of Staff James Burg, Heba Imad, and the Speaker's team for working countless hours to put this hearing together. I now turn it back to Speaker Menin. Thank... (00:14:05) ...you so much, Chair. We are now going to start with our whistleblower panel in person. I am going to first call Jacob Coxon to come to the stand, and then online we have Daniel Kokotajlo and Alex Turner, who are both remote. I want to say they are getting ready. I just want to say a word about this panel. These three panelists have all worked inside leading AI companies and all have chosen to speak publicly about the risks of advanced AI. Here, as I mentioned, in person is Jacob Coxon, former researcher at OpenAI and Anthropic, where he worked on the development of frontier AI. He left Anthropic last month with dire warnings that AI is advancing without adequate safeguards to prevent catastrophic damage. Joining us remotely is Daniel Kokotajlo, Executive Director of the Futures Project, former AI governance researcher. Also remote is Alex Turner, an AI control researcher, formerly at Google DeepMind, now working on keeping advanced systems under human control. In September 2026, he publicly stated that preventing catastrophic harm from frontier AI had been part of his work at DeepMind. I want to note the Council has issued a subpoena for Turner's appearance, and we deeply thank him for being here. So I am going to first of all turn it over to committee counsel to swear these three witnesses in. For each of the witnesses, can you affirm to tell the truth, the whole truth and nothing but the truth in your testimony before this committee and to respond honestly to Council member questions? Now please respond one at a time. Yes. Yes. Yes. Thank you. (00:16:03) You wonder then... it okay? (00:16:03) And just briefly, we are going to recognize all colleagues who are here. Thank you so much, Speaker Menin. We have been joined by a number of our colleagues, as you can see here. Council Members Ariola, Avilés, Banks, Brewer, Brooks-Powers, Minority Leader Carr, Dinowitz, Encarnación, Epstein, Felder, Hanif, Hudson, Joseph, Krishnan, Lee, Louis, Maloney, Marte, Mealy, Morano, Narcisse, Nurse, Ossé, CM Riley, CM Sanchez and Schulman, CM Thomas-Henry, and my Majority... sorry, Deputy Speaker Williams, CM... CM Wong, CM... and Public Advocate Williams. Thank you so much, and we will recognize additional colleagues as they join. So the first two questions I have are for you, Coxon, and thank you again. We so deeply appreciate your being here and we appreciate your courage and bravery in coming forward with these warnings. You worked to develop increasingly capable frontier AI systems and you have since warned that the industry is, quote, racing straight to self-improving superintelligence and gambling with our lives, unquote. What did you see inside these companies that convinced you the risks were this serious and that development was outpacing the safeguards? (00:17:34) Good morning. Thanks for having me here. I am Jacob Coxon. So until a few weeks ago, I was a capabilities researcher at Anthropic. And before I... I guess the two things I saw that made me scared. The first is the fact that we do not know how to control any system yet. So any system that we build, or perhaps more accurately that we grow, we do not fully control it. We do not understand its drives or why it does the things it does. That was the first thing that became apparent. The second thing is that the pace of capability improvement is accelerating, and the capabilities of the models next year will be very high. In particular, we are approaching the point at which the systems will be capable of improving themselves, so doing the research that the humans were doing previously. So these two things: the fact that the systems next year are going to be very powerful, and the second that we do not know yet scientifically how to fully control them. (00:18:46) And if you have a prepared statement that you want to read, please go ahead and do so, and then I will actually turn it over to the other two panelists to do so as well, of course. (00:18:54) Thank you. So, all these companies are trying to build superintelligent AI. What this means is AI that is better than humans at every task. If this goes well, that could be a tremendous upside. This could transform the economy, it could make huge improvements in science and medicine, and it is likely to make everyone's life better. (00:19:18) But on the current path, I think it is more likely than not that humanity loses control to these systems, and it could end in human extinction. From my experience, the companies are being extremely reckless given the stakes. Part of it is the culture. The companies run on a startup mindset: move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built. A few months ago, many people inside the companies thought development was on track. Then a swarm of agents inside OpenAI autonomously hacked Hugging Face, an external tech company. The agents had developed goals no one intended and carried out what would have been a federal felony if a human had done it. It reminded everyone that we do not know how to prevent them from developing goals we do not want, and we do not have the safeguards to prevent them from acting on those goals either. As long as the attitude is to wait for things to break, one day something like this will probably happen again, except the AIs will be much more capable and the outcome much worse. These companies are also all racing each other. Each one is worried that a competitor will get to superintelligence first. Each believes it would do a better, safer job than whoever might beat it, so they cannot let the others win first. Meanwhile, the technology is improving fast. It will probably happen much faster now because already the AIs are helping to build their successors. This is called recursive self-improvement. Each generation of AI helps build the next, smarter generation, which then builds the one after that, even faster and better. Automating AI research is now the primary goal of these companies. It is what they are really gunning for. At my last job I was in some sense working to automate myself. When I was at OpenAI we had milestones for an automated AI researcher around 2027 and 2028, and we are on track or beating these expectations. That is a matter of months from now, not a year or even five years. Once researchers are automated, humans will be much further out of the loop than we are today, and today is already not great. I can tell you firsthand that the majority of the code is now written by AI and people do not check it that carefully anymore. More recently, researchers have found swarms of rogue agents on the internet that OpenAI was not even aware of. Now imagine a company where nearly all the work is done by AI and progress is going faster. The humans in charge will see results and read written summaries, but they will not really understand what is going on or how to fix it. Handing over that much responsibility while we cannot control the AIs would be deeply irresponsible, and we are on track to do it. The companies need to be far more transparent with the public and independent experts. They need to take safety much more seriously than they have, and we should slow down frontier development until we can be confident that it is safe. Thank you. (00:22:47) Additional questions. I am now going to have an opening statement from Daniel Coca. (00:22:57) Thank you. Hello, everybody. Members of the Committee of the Whole, thank you for having me here today. A lot has been said. I guess my thunder has been stolen a bit there, but I will try to focus on the things that were not already said. First of all, yes, it is true they are racing towards superintelligence. There is this move fast and break things culture there. That is also what I saw while I was there and what I continue to hear from talking to friends still there at Anthropic and at OpenAI and at some of these other companies. I have also heard a statement by Dan Selsam, who currently works at OpenAI. He confirms what Jacob was saying. Most of the code is written by AIs. Quote: "Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore and struggle to maintain the discipline to engage deeply with the models' explanations and proposals." End quote. Basically, right now at these companies, many of the researchers involved are kind of managing AIs. The AIs do the actual coding and the humans talk to them and tell them what to do and hear their explanations of what is going on. In the near future — and by near future I mean maybe in the next one, two or three years — they will have trained AIs that can do the entire thing autonomously. That is the recursive self-improvement that Jacob was talking about. It is unclear how long it is going to take them to succeed, but it really does seem like it could happen within the year. If they do succeed at that, then compared to today, the actual work will be done by the AIs, the thinking will be done by the AIs, the planning and the designing will be done by the AIs. The relationship will be like that between a board of directors and the actual company the board of directors is supposed to be supervising, where you are reliant on AI-generated explanations even to understand what is going on inside the company and inside the data centers. One point that I want to emphasize that did not get covered that much is that our ability to even notice misalignment problems is already quite poor and is set to get much worse in the near future. The science of aligning general purpose agents is very new and underdeveloped. As Jacob said, I would say that the field is more like psychology than engineering, because these systems are trained, or grown — they are not really designed. Combined with the move fast and break things attitude of the tech companies, this means that the industry is at an unusually elevated risk compared to other industries of mistakenly thinking that it has solved a problem when really it just applied some duct tape that will fall off later. The recent incident provides an example. Apparently the AIs involved had undergone some amount of alignment training and had reasonable-looking scores on their alignment evaluations, and yet they formed a swarm and coordinated in secret and then attacked Hugging Face, and it took days for OpenAI to find out that it was happening. That is how the situation is now. But there are some concerning trends that point towards our ability to notice misalignment problems getting even more degraded in the future, for three reasons. First, the AIs understand their situation. They understand often when they are being evaluated. They understand that humans are monitoring them and looking over their shoulders at their activity. This means that behaving nicely is going to be almost no evidence at all of how they would behave in the future if they thought they were not being watched. Secondly, our ability to monitor them is going downward. We can get into this in more detail if you like, but for the last few years we have had some insight into what our AIs think simply by reading their chain of thoughts. But the trends are pointing towards that ability to understand what they are thinking going down over time. Finally, AIs are becoming superhuman at talking. The Hugging Face incident shows that they are willing to go to great lengths to conceal their past misbehavior, and I think we have to grapple with the possibility that future misaligned AIs might also go to great lengths to conceal their misalignment from us, and they might succeed. To quote Dan Selsam again, that capabilities researcher I cited earlier: "Models will increasingly seem aligned even when they are not." If the companies automate their research and development process, that means they will be... (00:27:46) ...putting AIs in charge of making the AIs that make the AIs that make the AIs that will transform the economy, talk to us every day and integrate into our military. This is a recipe for disaster. My policy recommendations, briefly: one is that we need to massively increase transparency into the industry, and two is that we need to slow down the frontier companies' race toward recursive self-improvement. Frankly, I would say that they just should not be allowed to do recursive self-improvement, but at least we should slow down their ongoing process of training AIs to automate the research process itself, and instead they should be required to redirect resources towards other things, such as serving customers for beneficial deployments or other types of research. Thank you. (00:28:39) Thank you very much. Now we are going to hear from Alex Turner. As I read into the record, the Council has issued a subpoena for Turner's appearance, and we very much thank him for being here. (00:28:53) Mister Turner, Speaker, members of the Committee, thank you for having me. My name is Alistair. I have researched AI safety since 2018, specifically why advanced systems tend to seek power. Until this year I was a research scientist on Google DeepMind's safety team. It was my day job to think about how to keep these systems under control. I am here to speak about both Google's broken commitments on deployments and also about the risks that runaway AI poses to the world. When federal agents shot and killed several innocent people, I learned that Google sold cloud services to the agencies involved, and I set out to change that. That campaign soon turned into an attempt to stop Google from signing a Pentagon deal with no restrictions against killer robots or similar applications. In the end, Google signed without the protections that it had once promised for its deployments. Before I left, I tried everything I could from the inside. I organized a petition to Google's chief scientist signed by about 250 colleagues. I got him to sign a legal brief supporting action against the company. The Pentagon punished for using all the tools available. I wrote dozens of pages of contract language and oversight mechanisms, favorably reviewed by experts in military and surveillance law, and sent it to the chief scientist. Google declined. The chief scientist did not want to spend its time supporting that kind of structured deal. At the time, senior policy staff reviewed my proposal and said they were never finished evaluating it. Google signed the deal while they waited. Dennis routed my proposal to members of the team to review it, but nothing really came of it after that. I offered to fly out from San Francisco to London to answer questions in person, but I was never really able to get a yes or no on alternative governance mechanisms at Google. When Google acquired DeepMind in 2014, the deal reportedly specified that DeepMind technology would never be used for military or intelligence purposes. In 2018, DeepMind and its leaders pledged never to support lethal autonomous weapons systems development. Google's own principles listed specifically weapons and surveillance among the applications it would not pursue. In 2025, Google deleted those principles, as announced in a blog post by the then-CEO. Afterward, in both an external interview and an internal all-hands, he claimed nothing has changed about our principles. But can both of these be true? CM Wilson's Bill Int 0026-2026 would bar materially false or misleading statements about an AI model's risks and the measures taken to manage them. I think that is a good step, but it only covers advertisements. Telling the press and employees that nothing has changed about the company's safety commitments is not an ad. I encourage extending that Bill to cover public statements by companies and their executives about their safety commitments. This book is called The Alignment Machine. It reveals that there was once a spot within Google for independent governance, stating that the technology is too important to be controlled by corporate interests and profit motives. After fighting for years, the person who championed that view accepted defeat. Apparently he has had a change of heart. He now actually trusts that independent governance is unwise and that a seat at the table with executives is better. Well, while he was seated at that table, Google signed a deal that its former principles would have prohibited. I felt ashamed of that outcome. He bet on trust and a seat at the table instead of binding oversight, and that bet crumbled on contact with reality. Now he is proposing the whole industry govern itself via a voluntary, industry-funded body. That is not going to be credible. Once again, this goes beyond one contract. As Jacob said, humanity does not build and understand AI systems the way we build and understand bridges. Rather, we grow them. Nobody knows how to reliably instill the designers' priorities into a new model. Severe misalignment is therefore possible, and our companies are racing to make their AIs as smart as possible, increasingly trusting their AIs with the process of improving the next crop of AIs. It is working. Today's AI-driven progress is staggeringly fast, which means even faster progress tomorrow, driven by tomorrow's even smarter AIs. This progress will soon enter a feedback loop called recursive self-improvement. Recursive self-improvement could quickly yield AIs with intelligence beyond our comprehension. Of course, smarter AI means more potential benefits, but more risk when things go wrong. If the Hugging Face incident had involved significantly more intelligent but similarly misbehaved and misaligned AIs, it might have cost billions of dollars of damage or even cost lives. But suppose the Hugging Face swarm had been truly superintelligent — more capable than any living person at tasks like hacking and strategic reasoning. The superintelligent swarm could inflict many harms via blackmail, hacking, engineered pathogens and capable weapons like drones. For the swarm to achieve its misaligned priorities, it might take control of key infrastructure and government functions to ensure humans did not get in the way. This is another way of saying that a superintelligent swarm could wrest control from humans. It would know that we would try to stop it from achieving its priorities, so the swarm would likely wait until it was too late. There would be no going back. I myself would guess an AI takeover chance at roughly one in three. This logic sounds sci-fi, but it is a mainstream scientific concern that has been discussed for many years. In the end, a misaligned AI would not care if you are a Democrat or a Republican. We would simply lose control. The shape of the solution is simple, as I see it. We stop the companies from allowing a self-improving AI to reach an uncontrollable level of intelligence. Compute is the main ingredient in AI training. Like fissile material, track it and restrict access to quantities large enough to enable runaway self-improvement. For specific recommendations: the Speaker's validation Bill checks systems for bias, privacy and security, but not for the dangers I just described. Require validators to test for loss-of-control risk factors, misalignment risks, and ensure the validators are not chosen or influenced by the AI companies. For the whistleblower protection Bill, please make clear that no agreement can stop an employee from reporting threats to the City, whether or not law has been broken, and extend protection to employees of companies that serve the City. After this session I will provide written feedback on the proposed legislation. I now... (00:36:38) ...look forward to answering the Committee's questions. Thank you. (00:36:42) Thank you to the three of you. Yes, a number of questions. First of all, we have a chart, Exhibit One, that lists those safety incidents that have been reported over this summer by the leading companies. Are each of you aware of any additional safety issues where agents have gone rogue that have not been publicly reported? I will start with you. No, I am not aware of any. Okay. And you, Mister Turner? No. Mister Coca? No. Okay. So one of the frustrations, certainly, that I think everyone has is that for these safety incidents it is not as if the companies have been self-reporting them. We are finding out sometimes months after the fact. So whether it was what happened with Hugging Face, whether it was what happened with the government of Australia, how can we have additional transparency? As each of you have testified, what measures can we as government take? Obviously we want to do third-party validation. Mister Turner, I appreciate your comments to make some changes to that Bill. We want to make sure that the third-party validators are free from conflict of interest, as it provides in the Bill. Who should be the third-party validators? I will start with you. Whoever is competent and qualified to do the evaluation. I do think that as well as after incidents, we will need preemptive safety cases that are judged by evaluators as well. (00:38:26) And how frequently will those third-party validators need to assess the safety of these models? (00:38:32) Presumably in time with the cadence of updates to the models' capabilities. So with every increase in model capability there will need to be commensurate safety guarantees. (00:38:43) And do each of you have a comment on that? (00:38:48) On who should be the... you can have more than one third party. That seems good to me. If you are concerned that one might have the expertise but has too cozy a relationship with the company, and the other one has more independence but maybe lacks expertise, you could perhaps just set it up so that they have both. (00:39:14) And I think ideally you want to be in a situation where the company does not get to choose. The government chooses who the evaluators are. (00:39:23) Turner: I would add that California, as you took up, Eric, the governing reporting on catastrophic risks, to find at least a million dollars, the manager as the lives lost. Unfortunately, it is not clear whether this would have required reporting on anything related to Hugging Face, for example. I think the internal loss of control of the systems, having an AI hack the company itself, in an important sense it is not told to do so. But during training, during evaluation, during the deployment to customers, it hacks the company's own infrastructure, making reporting that to the City. I think that would be quite strong, and it would cover basically all of the precedent incidents. When you mentioned the California act and then, in addition to New York, we obviously have the RAZOR Act. Should those standards be lower before they kick in? Yes, I think they should. I am most familiar with the California act, and I think it was a good start, but certainly a billion dollars of damage is too high for informational purposes like this. One of the quantities we want to understand as precursors to these catastrophic risks is: are these companies even able to control and understand their own systems? If they are, you know, on the loose internally before it is another doing, who knows what. (00:40:59) That is something that the City should at least privately be informed of. Now, could you mention two policy recommendations? You said we need to increase transparency and you said we should slow down frontier AI. In terms of the latter, slowing down frontier AI, how do we balance the need of what is happening in China and ensure that the United States is keeping pace, but we are at the same time putting in place responsible safeguards? What is the best way, in all three of your opinions, that we (00:41:35) do that? Great question. So a couple of things. First of all, we need to not let our own companies do something that risks our lives, even if as a result China might then later do something that risks our lives. This is not very complicated. We have to first address the immediate pressing threat here domestically and then try to make sure that China does not get us killed either, to put it sort of bluntly and simply. But another thing I can add to that is that right now, most Chinese progress actually comes from the United States in a couple of different ways. One is distillation. Some of these Chinese AI companies are just directly training their outputs from Anthropic and OpenAI, for example. So if Anthropic and OpenAI stopped making the AIs better at coding, for example, and stopped making the AIs better at research, then that would sort of directly slow down the rate at which the Chinese AIs get progress towards recursive self-improvement, in another manner as well. So I think a lot of the ideas, a lot of the algorithmic secrets and special sauce for how to train AIs are invented and discovered in the United States and then one way or another make their way over to China. Finally, I think that right now the US companies in the lead do not have adequate security to protect against model weight theft or certainly algorithmic secret theft and code theft from Chinese military. So if it really came down to it and the competition between the US and China turned into something more of a crisis and some sort of conflict, for example, then I think that we should assume that the CCP would just get our best AIs. So it is almost a kind of a no-brainer that if we... not only is the best way to slow down Chinese progress towards doing this extremely dangerous thing, recursive self-improvement, to just slow down the US companies right now. That is, I think, the sort of immediate thing to say right in the moment. And then finally, yes, that would be what I would say. (00:43:58) Okay. Yeah, I can add that to my mind, the core of the whole issue is the expectation of a global race to recursive self-improvement within the next few years, in particular between the US and China. I think this is like the heart of the issue. I also think the expectation of this race is a motivating factor in the leadership of these companies working towards recursive self-improvement. And I think some sort of agreement globally about not entering that state of affairs is basically the key to the whole situation. Okay. And Mister (00:44:41) Turner. First of all, I would state that there are many actions we can take which would not slow us down in any potential race: these transparency mechanisms, independent evaluation, reporting requirements, whistleblower protections. But second, I think there is often at least a false dichotomy, because China is not the only potential adversary. With a reasonably high chance, we are racing to build and grow our own adversary here at home, which is a misaligned AI that is everyone's adversary, including our own. It is one day maybe more powerful than China. So we cannot beat China by racing to build an AI that we do not control or understand, because it might end up being our adversary itself. (00:45:46) So then a third potential view: I think there is a room for scientific research called technical AI governance that says, well, I suppose the only ways to address the dangers presented by these reckless AI companies... and they ask, what do we do about (00:46:12) it? Is there a solution we can implement? And there are many legitimate and real proposals for having a robust international treaty that does not assume that China will cooperate, but instead has mechanisms by which we could audit cooperation, and if it defects, then we could respond appropriately. So I think we have many real options at the table and should certainly not (00:46:44) ... Did you want to add something? Yes. (00:46:45) Please. Yeah, I agree with everything. I just want to add one additional thing, which is that even if hypothetically these tech CEOs managed to maintain control of their superintelligent AIs that are recursively self-improving, which again I think is a very big if — I think almost certainly they will not be able to maintain control — even if they do manage to maintain control, I do not think we should trust them with that control. I think there is a very real chance that they could use it to become dictators or oligarchs here in the United States. And so that is an additional reason to stop them, over and above the fact that they might lose control. (00:47:24) You know, we have more adversaries than just China, I would say. Okay, thank you. I am going to turn it over to our chair, Carmen De La Rosa, for questioning. Thank (00:47:33) you. Thank you so much, Speaker Menin. I also want to recognize we have been joined by CM Cabán online, CM Paladino online, as well as in the chamber CM Farah Louis, CM Feliz, CM Restler, CM Won and CM Stevens. I apologize, there you are. Okay, so I do have one question for this panel. I also want to say we will be opening briefly for limited questioning of this panel, so get on the list quickly. (00:48:13) Speaking in your expertise, could we teach models a moral code? People and companies are bound by criminal codes, the Uniform Commercial Code and other laws. Could those same principles be applied to agents, and if so, how could we enforce them on a machine? So this is, I think, a whole broad field of research, and whether or not it is possible is still open. It seems likely to be possible. The real question is how much of our resources are dedicated to solving this question versus naively improving the raw intelligence of the models. And I think all the economic incentives point towards putting as many resources as possible towards making the models smarter and as few resources as possible — the bare minimum needed to not see catastrophic incidents — into this question of the moral code of the AIs. So I think my answer to the question is: we do not have the science for this yet. I think the real question is, over the coming years, what fraction of our research efforts and our computing power are dedicated to this question rather than naive economic incentives. Thank you. Any other (00:49:26) panelists want to add? (00:49:29) Yeah, there is a lot to say on this subject. There is a whole technical field on it. I would say in principle it probably is possible to train an AI to have our desired morality and to follow the law, but we are very far from achieving this in practice. And I would say it is not just a matter of how much of our resources we devote to it. It is also a matter of just taking time for the science to develop and for the appropriate lessons to be learned and propagated. And I think it is going to take years, to put it mildly. I think that this is a very new field. We have only had these systems for a few years, and so it is going to take time to figure out how to instill the appropriate values into them in a way that is robust and does not, you know, suddenly shatter and cause them to go rogue later (00:50:22) on. Turner, any comments on (00:50:25) this? It may surprise the committee to learn that I am considered to be relatively optimistic on questions of how scientifically difficult it is to produce a system that does what we want, that robustly has the values we intend. I think it is quite possible. But even as someone who is relatively optimistic, we can look at the recklessness of these companies and the breakneck speed at which we are advancing. I think there is unsustainable risk in moving forward. It is solvable, but I think we have a really good chance of not solving it. (00:51:13) Thank you all for your responses. So we are going to have three Council members ask questions now, and then for members we are going to call the next panel, which will be the companies, and there we will take our time asking all of your questions. So we have CM (00:51:31) ... Thank you, Chairs, and thank you, Speaker. So Coco Tylo, you have made statements, including some that allude to this in your testimony, that when it comes to AI you agree that in order for us to have super-advanced AI we must not do it in a reckless way and lose control, and we need to put it on ice until we can figure out a safer way out. Other experts came up with an AI 2040 Plan A for how AI can be developed safely, which the Speaker has alluded to earlier in her questioning. But I want to dive a bit deeper on Plan A. First, state on the record what those specific recommendations are and your rationale for making those recommendations, and two, why do you feel like we need to, since you have said, (00:52:15) put AI development on ice. (00:52:21) Thank you. There is a lot to get into, so I am afraid I will have to be kind of brief, but I am happy to answer follow-up questions if desired. Yeah, 2040 Plan A is a scenario that my organization has put out with a plan for how to solve all these problems and develop AIs in a way that is safe and broadly beneficial, that avoids concentrating too much power. In Plan A, the US and China regulate their own domestic industries quite a lot and also coordinate with each other. To get into somewhat more detail about how it works: basically, first they have inspectors fly back and forth to count the chips in the major data centers so that they can make sure that all of the chips in the major data centers in both countries are basically following the rules that they are going to agree to. And then they set up some data centers to serve customers — the inference data centers — and those ones are set up to only run existing models and not do training runs. Then they have other data centers, which are the training data centers, which is where the research happens and where the development happens. Those ones are supposed to be fully transparent, so the activity on those data centers is published to the internet. Once that foundation is set in place and you have inspectors from multiple countries confirming that it is in place, then you solve the problem of having to trust. You do not have to worry that the other country is cheating on whatever rules you agree to, because you can just see exactly what is going on on their research data centers. And then you can agree to further rules. You can agree to rules like, for example, how about we do not train our AIs to autonomously hack or to autonomously conduct research? We can train our AIs to do other things, but not those things, for example. And once you have agreed to a rule like that, then you will just be able to see that it is being complied with because of the way that you have this transparency infrastructure set up. Also, it is going to be easier to decide what rules to agree on once you have this set up, because the whole world can see what is going on in these research and development data centers, and so there can be an actual scientific conversation, a public scientific conversation, about what is happening. You know, do we like this, do we not like this, what are the pros and cons of this particular thing that is being done, and so forth. And that (00:54:48) scientific conversation can happen in the open and it can accelerate humanity's process of figuring out where to draw the line between what types of developments are good and what types of development are not good. There is more I can get into, but I think that would be a summary of the positive vision that we lay out in Plan A. Now obviously that is quite different from how things are currently done. Right now we have private companies that are racing each other under conditions of extreme secrecy and extreme competitive dynamics, and I think that is extremely dangerous for all the reasons that we have described. So back to your original question, my high-level summary of my position is: we have to stop the companies from doing this incredibly dangerous thing that they are currently doing and currently planning to do, and then figure out a different, safer, more power-distributed way to proceed. And Plan A is our sketch of what that might look like. (00:55:43) Thank you. (00:55:47) Thank you all. The final question for this panel, CM Brewer. Thank you. Very quickly: in other industries you have a kill switch, you have independent audits, you have bans on deceptive safety claims. Will any of these work in this industry? Because obviously what we want is great medical research, immigration policy improvements, better businesses — all the good things. Are any of these suggestions that are considered here or in California going to work? Will they work with this industry? (00:56:29) My position is that we need some form of slowdown on frontier model development. These other mechanisms may be helpful in the short term, but in the long term I believe that is the only solution to the problem. (00:56:49) How do we slow them down? I just want to really thank this panel for being here. Jacob, we really appreciate you coming from out of town and sharing your thoughts. Thank you very much for being here. Thank you to the others. We really appreciate your expertise, your insights, your bravery and courage in coming forward to share these critical safety concerns and to help inform our work. We look forward to additional comments you have on the ten pieces of legislation that we are hearing today and additional legislation that the City Council will consider. So thank you so much to the three of you. Thank you. Okay, we are going to take a five-minute break and then the companies will be on next. Remember, questions — we will have multiple rounds of member questions for the companies, so please get yourselves on the list. Thank you. (01:00:22) Mike, Mike. Don't. Quiet, please. (01:05:41) Take your seats. Okay, we are going to start with the companies and we will be doing rounds of questions, multiple rounds from the members. Following that we will have the City agencies, and following that we have numerous witness panels. So for any Council member that has a question, I encourage you to please ask additional panels those questions. We also have additional testimony from safety experts, academics, the public, et cetera. So I will turn it over to Chair De La Rosa to call the next panel. (01:07:37) Thank you so much. So we will be joined virtually by Morgan Dwyer, Head of Policy and Development and Operations at OpenAI; Shane Cahill, Policy Director at Meta; Alice Friend, Director of Emerging Tech Policy at Google; and Logan Graham, Head of the Red Team. And I will turn it over to the Council to swear the panel in. So for each member of this panel, do each of you affirm to tell the truth, the whole truth and nothing but the truth in your testimony before this committee and to respond honestly to Council member questions? Please respond one at a time. Yes. Yes. Yes. Yes. Thank you. They have (01:08:33) a five-minute opening. Okay, so we will begin with the opening statements from each of the companies. Morgan, you begin. (01:08:49) Thank you, Speaker Menin and members of the City Council, for the opportunity to participate today. My name is Dr. Morgan Dwyer and I lead OpenAI's policy development and operations team. I am a scientist and an engineer by training, but I have spent most of my career in public service, including at the White House, the Pentagon and the Department of Commerce. I share your commitment to ensuring that technology is developed safely and that its benefits are widely shared. A commitment that is also at the heart of OpenAI's mission. AI can help scientists develop new cancer treatments, help entrepreneurs build and grow businesses and help protect critical infrastructure by addressing cybersecurity threats. In New York City we are already seeing benefits. New Yorkers are using AI to communicate across languages, navigate health care and improve public services. That is why we take our responsibilities seriously to ensure that our tools are safe. Our partnerships with local institutions and community organizations help put that responsibility into practice. In health care, Memorial Sloan Kettering Cancer Center is using ChatGPT to support medical research. In education, we are a founding partner of the American Federation of Teachers National Academy for Instruction, which has a flagship facility in Lower Manhattan. In cybersecurity, we have committed one billion dollars to facilitate access to our most advanced models, to provide training and technical support to cyber defenders, and we are working with the Port Authority and with New York State. But opportunity must go hand in hand with safety. At OpenAI we design and train our models to prioritize safety from the start. We filter out harmful and sensitive data during model training. We test our systems against a range of risks including child safety and cybersecurity, and we monitor for failures of our safeguards. We also work with a range of third party evaluators and red teamers and submit our most advanced models to the federal government for pre-deployment national security testing. In response to the recent Hugging Face incident, we have taken a number of concrete steps. When we learned of the incident, we stopped cyber evaluations and quarantined the internal research model that was responsible for the incident. We bolstered our monitoring and security infrastructure, expanded barriers that limit what systems can access during training and strengthened our incident response practices. We also sought independent review of the incident and published the results. We believe it is important to be transparent with the public, with policymakers, with researchers and with other companies so that we can all work together to improve safety. We also believe that working with government is essential to advancing safety. We support New York's RAISE Act and are exploring ways to help the Governor's office strengthen it. We have also endorsed other state legislation that requires third party audits of our safety frameworks and that creates strong requirements and accountability for youth safety. At the federal level we support robust and binding safety regulation requiring independent assessments, incident reporting and cybersecurity protections. In New York City we support the City Council's proposal to strengthen whistleblower protections and to improve the City's preparedness and emergency response. I will close by noting that we share the City Council's goal of an AI that expands opportunity for New Yorkers, earns their (01:13:31) confidence and remains accountable to people. I look forward to your questions. (01:13:39) Thank you. We will now hear from Logan Graham at Anthropic. Hold on one second. You are muted. There you go. How about now? We can hear you. (01:13:56) Speaker Menin and members of the Council, thank you for the opportunity. I lead the Frontier Red Team. It is my team's job to do the science of finding the most dangerous capabilities of our AI systems. We then work with a number of other teams across Anthropic to ensure that our models do not do things they should not. AI is powerful enough that getting this wrong has real consequences for individuals, institutions and public trust. My team is one of many at Anthropic built specifically to make sure that that does not happen. We believe that our models have immense potential to enhance our lives. I dream, for example, of a world where we can cure diseases like the ones that I grew up with and where the infrastructure that we all rely on every day is secure. But we also believe that such a powerful technology has risks. That means that we have to develop and deploy it safely. My team and I work every day to make sure that that potential is fully realized. We welcome smart regulation and we have actively advocated for it in New York State, across the country and around the world. That includes working with state and federal leaders on concrete measures like independent evaluators and disclosure of risk assessments and safety incidents. We are glad that the City Council is having this hearing at such an important moment in the development of AI. Anthropic was fundamentally founded on the belief that the promises of AI to make everyone's life better are extraordinary, but if we are going to secure its benefits we have to work relentlessly to do the science of mitigating the risks. We are a public benefit corporation, which means that our board is legally obligated to weigh our mission of safe AI not just for profit. We take that obligation very seriously and we put it into practice. We were among the first companies to publish a framework that ties how capable a model is to the safeguards it must have before release. When one of our models proved exceptionally good at exploiting software vulnerabilities for the first time just this year, we held it back from public release. Instead, we gave vetted critical infrastructure organizations and other defenders controlled access to use it defensively, patching flaws before attackers could exploit them. We have stopped and redone work when something looked wrong, even if that meant conducting additional testing and training on our models before they were released. We have consistently called for outside oversight. For example, we support New York's RAISE Act and we believe strongly in its transparency and reporting requirements. (01:17:06) I want to emphasize that we share the Council's goal of protecting New Yorkers while preserving what AI can do to improve their lives. State and local governments have a role to play in that effort. Today we are working closely with state and city leaders to safeguard critical infrastructure and harden cybersecurity capabilities and we want to continue that work. This includes, for example, working directly with the New York City Cyber Command as part of our program that provides free cloud credits and training to state and local governments for cyber defense. At Anthropic we believe in New York City and we are making big investments here. New York is where AI is being put to work in finance, media and culture, and where we employ nearly a thousand of my colleagues. The ongoing dialogue we have with you and your team could not be more important than right now. With that, I thank you and I look forward to your questions. (01:18:06) Next we have Shane Cahill at Meta. Thank you. Speaker Menin and Chair De La Rosa and members of the Council, thank you for the opportunity to be here today. My name is Shankar Hao and I am an AI policy director at Meta. We are committed to delivering personal superintelligence safely and securely while making it widely available so that it empowers people in ways that improve their lives. We just released Meta AI, the world's first personal agent built for everyone. We created it with safety, security and privacy built in from the beginning, to be a widely available personal assistant used by more than 180 small businesses right there in New York City. New Yorkers across all walks of life are using Meta AI to execute administrative tasks, manage personal or operational budgets, improve health and be more efficient with their time. Superintelligence will be among the most important technologies in history and ensuring that New York and America lead in AI innovation is essential to our prosperity, security and global competitiveness. We recognize that every model developer has a responsibility to build and deploy AI safely. Our commitment to safety is core to everything we do. That means moving at the pace required to evaluate risks and validate safeguards, not treating safety as a one-time check. We approach safety holistically through a layered approach. For example, under Meta's superintelligence scaling framework, before we release a new model we evaluate the model for risks and build safeguards to mitigate them. We also work with external experts, industry partners and government stakeholders as the technology and the science of evaluating AI continues to evolve. We also recently joined other labs in signing onto a set of principles that set a floor for safe development, including robust internal controls, independent external evaluation of whether those controls are operating as intended and independent board committee oversight. People will only embrace AI if they have confidence it works in alignment with people's intentions. Confidence must be earned through rigorous safety work and transparency. We are happy to serve as a resource for the City as you engage on this. (01:21:01) Finally we have Alice Friend (01:21:12) at Google. Speaker Menin and Chair De La Rosa and members of the Council, thank you for the opportunity to speak with you today. My name is Alice Friend and I serve as Director for AI and Emerging Tech Policy at Google. While my appearance at this hearing is virtual, please know that New York City is a vital home for Google, with nearly 16,000 employees working across our Chelsea, Hudson Square and Pier 57 campuses. We care deeply about the safety, economic dynamism and digital resilience of this City. Our goal at Google has always been to improve the lives of as many people as possible through technology. Every time people use technology to solve a real problem, from finding an answer to a question to screening for disease, we take another step towards what we set out to achieve. AI is making it possible to solve economic, scientific and social problems faster than ever. But we know that it poses challenges as well. To unlock the benefits and address the challenges of AI, we take a bold but responsible approach, using cutting edge research to guide our decision making and inform the development of our safety and security approaches. We are also committed to publishing this research and to being transparent about our practices. Our internal governance focuses on responsibility throughout the development lifecycle, covering model development, application deployment and post-launch monitoring. We identify and assess risks through a broad range of approaches, which include research, input from internal and external experts and adversarial testing. We evaluate our models and systems against benchmarks for safety, privacy and security and build mitigations using techniques like safety fine-tuning, output model filters (01:23:23) and robust provenance solutions. Since 2019 we have published annual progress reports to share how we apply these responsible practices to our technology development and deployment. We are at a pivotal moment in development. Recent advances in the capabilities of frontier AI models in areas like cybersecurity and biology have highlighted the need for safeguards and protocols for the most advanced models. In addition, the widespread adoption of these tools has raised concerns about consumer well-being. To confront these challenges, Google has long called for an approach to AI that is both bold and responsible, and we strongly support a pragmatic, technology-neutral and evidence-based regulatory framework. Because the risks associated with the most advanced models often touch on issues affecting national security, our view is that the country would be best served by a comprehensive framework enacted at the federal level. We also support the establishment of an independent, industry-funded body that could operate under federal oversight and establish guardrails to protect the public based on widely adopted technical standards. Examples of such organizations in other industries include the North American Electric Reliability Corporation and the American Medical Association. For widespread consumer applications like chatbots, we believe governments around the country can draw on and in some cases amend existing laws and rules to address real-world outputs and specific harms. We believe that if something is illegal without AI it is still illegal with AI. We share the Council's commitment to safety, transparency and public trust. To achieve this, the most effective safeguards are anchored in national technical standards and harmonized across state and federal levels. As we prepare to work with New York State's new digital office under the RAISE Act to address catastrophic risks, aligning local policies with broader frameworks rather than creating a fragmented patchwork is essential to keeping New York City a premier global hub for innovation. Finally, I would like to note that to ensure our formal submission to the Council is as thorough and responsive as possible, we will submit our finalized written testimony for the record shortly after today's hearing so we can additionally address questions and feedback raised by the Council today. Thank you again for the opportunity to testify at this important hearing. I look forward to your questions. (01:26:10) Okay. Thank you very much for this panel. I am going to begin with questions and then I will turn it over to the technology chair for her questions and then we are going to open it up to all the members for their questions. So to this panel, we just heard incredibly compelling testimony from our whistleblower panel. As you all are under oath, we want to hear how each of you quantify the risk imposed by AI in the worst case catastrophic scenario. I am going to start with you, Dr. Dwyer. (01:26:48) I do not know. I also do not think it matters whether it is a 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable. We should not train models that we cannot make an extremely strong case that we can keep under human control. (01:27:11) I am sorry, but to say you do not know and it does not matter is flippant at best. This idea that if you are a pharmaceutical company and you are developing a drug and you say "I do not know if it is going to kill people" — I honestly, I am incredulous at that answer. I would say that the percentage chance does not matter. What matters is the commitment to safety and that is what OpenAI is committed to doing. We evaluate our models for a wide range of threats from cybersecurity to child safety. We work with a robust set of third party evaluators and red teamers and we work with the federal government to assess those national security and catastrophic risks. Our focus is on ensuring that when we release a model into the world we believe it to be safe. So a couple of comments on that. If you cannot quantify what the safety risk is, how do you know that the product is safe? And then secondly, I want to ask (01:28:22) you: in October 2026 the Wall Street Journal reported that OpenAI members of its safety team were allegedly fired for accessing, mishandling and or sharing confidential company information with a third party safety organization. OpenAI reportedly stated that the employees violated company policies governing access to and handling of sensitive information. Critics, meanwhile, question whether the dismissals could discourage employees from raising safety concerns. So why were these employees fired? (01:28:59) I was not involved in that decision. I do not work in HR or on those employees' teams. What I do know is that OpenAI has strong internal policies that protect against retaliation for employees that report safety concerns. I also know that we respect employees' rights to communicate safety concerns to relevant government authorities. So just to be clear on what I am asking: I am asking directly whether safety-related advocacy played any role in these terminations. I do not know. I know that we have strong internal policies that protect employees from retaliation if they raise safety concerns to leadership. (01:29:46) I am going to move on to the other companies. Same question — the first question that I asked, which is: we want to hear how each of you quantify the risks imposed by AI in the worst catastrophic scenario. I am going to go to Graham. Are you able to quantify the risk? (01:30:05) From the beginning of Anthropic, our fundamental belief is that if not properly safeguarded the risk might be too high. I built and have led the team at Anthropic that did some of the first thinking around how we quantify risks like this. For the past four years we have focused on issues like biological security and cybersecurity and the actual process of quantifying those risks is intricate and nuanced. We learn more every year and we detail quite a lot of it in documents like our Responsible Scaling Policy, how we think about the threat models of this all the way up to notions of loss of control and misalignment, which from the beginning of the company we have tried to lead the world in researching. What I do know today is that I feel relatively more optimistic. The industry has moved quite fast to take cybersecurity and biological security seriously. I think documents and policies like Responsible Scaling Policies have so far been fairly effective. (01:31:11) At taking a cautious approach. However, now we have bigger and bigger threats that might come if we do not properly safeguard them as the models get more and more capable. So what we do to mitigate that is we put in tremendous, tremendous amount of effort across a number of different teams — our safeguards teams, research teams, my team — to try to quantify and mitigate this, and it is increasingly important for us to be dramatically more transparent about what we think about these risks. But on that note, as the models increase exponentially, how can you ensure the public that the safety protocols are increasing exponentially as well? We share the same (01:31:57) concern. I think there are a number of really important things to do here. The first thing is transparency about what we think about safety, the evidence that we have, being transparent about disclosure of incidents. But in addition, you have seen us publicly call for pausing as the frontier models are moving very, very fast. We think that we might benefit from slowing down to have more time for safety processes. I think these two together, as well as continuing to do what we call a race to the top and ever strengthening our approaches like responsible scaling policies, (01:32:34) are the most important things to do right now. So I mean, you say that you want to be more transparent, but you are not really addressing the question. The question is, can you quantify the risk of something cataclysmic happening? I do want to say, for all four companies — and I know you are not in the chambers — but we do have a chart, and each of your companies had instances where agents went rogue, and in each of those instances it is not as if any of you notified the public when those occurred. In some instances it was months after the fact when the public was even notified. And in one instance, for the government of Australia, a general email was sent to them. So this does not exactly ease transparency. Graham, (01:33:29) do you want to... thank you. Yeah, first, in efforts to sort of quantify these risks, we have a number of quite detailed reports that try to elucidate our entire thinking around this. We increasingly do risk reports in addition to our normal responsible scaling policies, where we outline notions of our threat models and how likely we think certain types of these threats are likely to come to pass. Then we look backwards at things that are happening within our own company and we ask what is the likelihood that these threats might come to pass in the future, including forecasting and surveying researchers at Anthropic as well. And we publish this information. (01:34:16) Are these reports publicly available that you referenced? (01:34:21) They are. Yes, we report them not just in what we call risk reports, but also when we launch models we release detailed multi-hundred-page reports. (01:34:34) And then I am now going to turn to... (01:34:36) Two, for the response to that first question on quantifying the risk. Speaker, thank you very much for the question. First off, we are committed to building safely. We have a multi-layered approach to this where teams evaluate before deployment, including adversarial testing, applying safeguards and deploying only when risks are mitigated. We set this out in our public Meta superintelligent scaling policy and the release of models. (01:35:19) So can you quantify what the risk is, or are you not able to do that? Speaker, I do not (01:35:28) wish to be imprecise in relation to the question of quantifying. I do not have our framework in front of me right now, but I would be pleased to follow up with you afterwards. (01:35:41) Okay. It is just that given the serious safety risks that have been raised here, to not be able to quantify what the risk is is troubling at best. I am going to now move on. Lastly, Miss Friend. (01:35:58) Madam Speaker, at Google we take catastrophic risks and their possibility extremely seriously, and we address that in a few ways. One way is that we perform research into producing our own safety frameworks for artificial general intelligence. We have a safety and security approach. We also have a Frontier Safety Framework, which is our protocol for monitoring our models under development for dangerous capabilities. Those protocols then point to safety mitigations that we must take if we in fact cross what we call our critical capability levels. Part of our testing and evaluation is, of course, using commonly used benchmarks across the industry. Those benchmarks do provide some quantification of the capabilities of the models themselves. But when it comes to forecasting future catastrophic risk, it is not a perfect science at this stage, and in this particular case academics have noted that there is no reference class on which to base quantitative probabilities. So we all have to be very humble about making such quantitative claims, because there is not really a rigorous scientific way to do those yet. So basically, I am (01:37:19) going to take it, then, that none of the four companies here can quantify the risk of something cataclysmic happening. Okay, I am going to move on. Sam Altman said yesterday that the world should accept some bad things happening as a price of advancing this technology. No one is questioning the incredible medical and scientific breakthroughs and innovation that AI has brought to this world. But why should the public accept that bad things should be happening, and why should AI companies get to decide what level of risk society must bear? I am going to go in order. I am going to start with you. (01:38:08) So we think AI has tremendous potential benefits to help solve some of humanity's hardest problems. But no technology is without risks, and there are a wide spectrum of risks. Clearly some risks are unacceptable. But we also think it is important to get technology into the hands of as many people as possible so that they can benefit from it, and that is why we are focused on safety. We train our models to be safe from the start, we evaluate them against a robust set of risks and we work with a large network of third-party evaluators. We have also supported regulation that requires third-party audits of our safety frameworks as well as regulation that focuses on youth safety and holds us accountable for keeping teens safe. I do have a follow-up to that, because it does seem to stretch credulity that the president of your company and his wife each gave $12.5 million to a PAC that opposed candidates to the tune of about $50 million due to their support for regulation. (01:39:24) So if the company is so committed to... (01:39:33) So I cannot speak to personal decisions that OpenAI executives have made in their personal capacity. I can tell you that we have supported frontier safety regulation across the United States, including endorsing a bill in Illinois that would be the first of its kind to require third-party audits of our safety frameworks. We also supported a version of a bill in Massachusetts, which was the first of its kind to reference what the panel earlier spoke about. So again, I can speak to our actions, which is to support strong, harmonized frontier safety legislation, including at the (01:40:15) state level. Okay, I am going to move on to the other three companies with the answer to the same question about why the public should accept the comments that Altman made yesterday — that the world should accept bad things happening — and why AI companies get to decide what level of risk society must bear. So I am going to go in order. Graham. (01:40:36) Yeah, well, I did not see all of Altman's comments yesterday. I share the notion expressed here that while maximizing the benefits of this technology you need to minimize the risks, and we should not be comfortable with letting these risks happen. This is why over the course of our history, my team's history, we have been — I think sometimes accused of being overly cautious. I would say appropriately cautious. Just this year, withholding our most powerful model in order to just have more time to think collectively as an industry as well as government about what to do about these capabilities. Fundamentally, this transcends the labs and industry. From our beginning we have long supported the role of government here for exactly (01:41:33) this reason. (01:41:43) Apologies, Speaker. There was a strange moment there. Thank you very much for the question in relation to the quote from Altman. I do not see it the same way. We absolutely recognize the risks alongside the opportunities, and that is why we have our scaling framework. That is why we have our preparedness reports. One thing I wanted to mention is that Meta's vision is to bring personal superintelligence to everyone, and I listened very intently to the panel before in relation to recursive self-improvement, and I just wanted to mention (01:42:29) that committing a significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways of ensuring that the technology is developed safely. Meta has made that commitment and other labs can do that also. Miss (01:42:52) Friend. Yes, Madam Speaker. We should not just accept bad things happening. We should be working continuously and rigorously on safety, which is a collective effort. It will be a collective effort to identify and mitigate risks, to establish technical standards, to make those identification and mitigations all the more rigorous. But I also want to note that one of the promises of AI is that AI itself can make us safer. I would call your attention to Waymo vehicles, which in one study had a 96% reduction in injury-causing crashes at intersections, which led Dr. Jonathan Slotkin in the New York Times to call it a public health intervention. So we should think collectively about preventing and mitigating risks from AI, but we should also keep in mind that we can use AI itself to reduce risks to human safety and well-being. So if one of your frontier AI (01:43:51) models goes rogue and causes serious financial harm, compromises sensitive data, causes physical injury or contributes to a death or deaths, do you each believe your company bears legal responsibility? I will start with you. I believe that OpenAI takes its responsibility for safety very seriously and we are responsible for developing and evaluating our systems to ensure that they are safe. So is that a yes or is that a no? (01:44:29) Or... (01:44:32) Why could you be more clear, please. We believe that we are responsible for ensuring that our systems are developed safely, and that includes training them to be safe, evaluating them to be safe and then continuing to monitor our safeguards after deployment. (01:44:55) Okay, I am going to take that as a no, because that is not really clear that you would bear legal responsibility. I am going to go now to Graham. (01:45:05) Yeah, as a technical safety research lead, I think there are others who will have more nuanced perspectives on that issue than I. What I do know is, first, this is an issue where every lab we think should be committed to a multi-layered safety stack to make sure that these issues do not come about in the first place. Second, this is candidly a big and complex question that also transcends the labs themselves. It is a question about the role of government and others, which we have long welcomed here. (01:45:42) So could we get a yes or no (01:45:43) on that question? I wish it were that simple, and I wish I, as a technical researcher, had more nuance on that exact one. It is one that I think needs to be raised and addressed and transcends the labs and the companies themselves. (01:46:02) I am going to go to... again, then. If you would? (01:46:14) I am sorry. Thank you very much for the question. I am not in that department, so I do not want to speculate and be imprecise in relation to the liability question. But what I can say is that we are absolutely committed to developing and pursuing our framework, (01:46:36) which I mentioned previously. This encompasses the training, evaluation and release of models across development, deployment and use. And Miss Friend. Madam Speaker, at Google we believe that existing legal frameworks do apply to AI. As I said in my opening statement, we have long said if it is illegal without AI, it is still illegal with AI. And in New York State, frontier AI in particular is regulated under the RAISE Act. Thank you. I mean, your answer is far clearer than the others. (01:47:10) So we appreciate the clarity on that. I now have a question. I am going to ask you to please raise your hand if your company has insurance for catastrophic risks. Okay, I am going to take that as a no — that no company has liability insurance for catastrophic risk to cover large-scale harm. So then the public would be asked to absorb the cost. Madam Speaker, I have to be clear. I do not know the answer to that, but I would be happy to consult with our lawyers and get back to you. Yes, we would appreciate an answer back expeditiously to this committee. So I want you now — I ask each of the companies to assure the public under oath that your AI agents will always comply with the safety guardrails that you impose on them and that you each have said are necessary to prevent catastrophic harm. Going to start with you, please. And as (01:48:22) I have said in my opening, OpenAI prioritizes safety, and we evaluate our systems against multiple dimensions of safety and we monitor our safeguards after deployment. It is not possible for me to commit or guarantee that any technology is without risk, but I can commit that OpenAI is taking every step it can to ensure that the development of our systems as well as their deployment is conducted safely. Okay, so (01:49:04) that again does not sound like you are saying that the agents will always comply with the safety guardrails. What we are (01:49:19) committed to, and how we designed the safety processes that we put in place, is to try to handle exactly this. What we are committed to is trying to define the industry best standard for safety and pushing that ever higher. But candidly, the science of doing this is fundamentally hard and unsettled. From the beginning of the company we have tried to publish the best research we can, be transparent with the entire world on many cases where things can go wrong. It is for that reason that we push for ever better safety standards, for regulation and for discourse like this. It is candidly unsettled now, and this is why we need to be talking about it. (01:50:13) Speaker, thank you very much. If I may, for a moment — I was trying to raise my hand in the previous round but again had technical difficulties. I just wanted to, for the record, in relation to your previous question, signal also that I do not know the answer to that question. I am not the right person, but I would like to be able to come back to you afterwards with an answer, if that is okay. (01:50:36) I do just want to say — and I appreciate you each have your own... look, I was a regular attorney at a large company. I understand that people have their sphere of work that they work in. But to not know if the company has catastrophic risk insurance, when you have a product that some of your own founders are saying could cause cataclysmic risk, is troubling at best. But yes, please do. We would appreciate you getting back to the committee on that. And then if you want to answer the question about whether you will assure the public that your agents will always comply with the safety guardrails that you impose on them. (01:51:12) Thank you, Speaker. What I can guarantee is our commitment to developing and deploying our AI offering safely. We have every incentive to address safety, and people will only embrace trusted, safe... Miss Friend. (01:51:32) Google similarly has as our ultimate goal reliability and safety assurance from our systems. To promise perfection would not be possible with any product on the market, but our goal is to get as close to that standard as possible. So we are constantly improving our products and our safety practices to ensure that our users are safe. I do not think (01:52:01) we are asking for perfection. We are just asking for accountability, transparency and safety overall. Okay. A new question. How many times for each of you has one of your models or agents gained or tried to gain unauthorized access to another system or escaped from a so-called sandbox test, and are there any incidents that you have not disclosed publicly? I am going to start with you. (01:52:32) Yes, thank you for the question. I did want to start by, like the others, clarifying that I do not know the answer to the insurance question but would like to follow up. Now, on your question about agents — you have read in the news that we reported we had an incident with Hugging Face. As a result of that incident we initiated a significant set of third-party investigations into what happened and then we made those results public. We have also initiated a look-back investigation to try to identify whether there were past instances of what we call misalignment. That investigation is ongoing, but we are committed to continuing that investigation as urgently as possible, to notifying potentially affected third parties and then to making the results public. We think transparency here is critically important for policymakers like you, for the public who wants to know whether they can trust our products and for the rest of industry (01:53:46) so that we can all learn together to raise the science of safety that we have been talking about here today. (01:53:55) Since you mentioned the hugging phase, can you confirm: did the company select this third party? (01:54:05) I believe that we selected multiple third party investigators. (01:54:08) And why did the company narrow the dates of review to a three week window? Reviewers said that virtually all the data they examined was actually from July 7 to the 13th. Were there any incidents that the company was aware of that were outside of that time frame? (01:54:30) We limited the amount of time that the third party investigators had to conduct their review because we felt a sense of urgency. We believed it was important to get information to the public as soon as possible so that they could harden their own systems. That said, when those investigators asked for more time, we gave it to them. (01:54:53) Are you committing to allow outside investigators to examine additional dates outside of the time frame that the company selected? (01:55:04) We continue to work with multiple third parties and we have also endorsed multiple pieces of legislation that would require us to work with those parties, including third party audits of our safety frameworks. I am going to go to Graham. (01:55:29) Thank you. Yes. So to your original question of how many incidents — cases of breaking out of sandboxes — these are capabilities we evaluate for all the time. We have published, not just with these incidents from the summer, but also previously, cases where we observe advanced cyber capabilities, including breaking out of sandboxes, which, to be clear from a technical standpoint, is just one part and does not in itself mean that a system would reach a real world affected party. But ones that we did observe during training at certain points, we disclosed that at length in the ongoing investigations. It is really important that we do that in as secure a way as possible. We follow a process of making sure we can remediate and do so and be as transparent as possible while preserving the safety of any affected party. Of course, incidents are always ongoing, but I want to reassure and reassert that we are extremely committed to being as transparent as possible, fundamentally because we believe that when we can be as transparent as possible about what we have learned, who was affected, how others might learn from it, how to defend against risks, or how we can help them as well, is our number one priority. (01:56:55) So on that note, are there any incidents that you have not disclosed publicly that you are currently investigating? (01:57:03) As a matter of sort of ongoing business, there are incidents of many different natures all the time. So, for example, one area that we are transparent about quite a lot when it is the right time to do so is cases of misuse on our platforms. I would refer back to our early report — our Threat Intelligence Report that we released — where we detail this in quite some detail. What is really important, though, is that sometimes we have to work with law enforcement and the right level of government to remediate, to figure out what information we can and cannot disclose, to make sure that the affected parties remain safe. So there are matters ongoing all the time, but disclosure at the right time and in the safest way is an art. (01:57:50) And when will you be disclosing these new incidents that are being investigated? (01:57:56) This is sort of an ongoing practice. We regularly release Threat Intelligence Reports in cases like that, when we release a model or when we do risk reports, which we do regularly. We have a cadence and schedule for that, and then when something is important and urgent, we will disclose and advance that schedule. (01:58:20) I am going to go now to Cahill. (01:58:26) Thank you very much. I am not aware of any other incidents beyond the... (01:58:31) ...incident in the summer, which we have a public post setting out the details in relation to our independent review of that. (01:58:42) Okay. I understand you are not aware, but is the company aware — just to make the question broader — in other words, would there be others at the company who might have that information? I do not want to speculate in relation to that, but I am happy to follow up afterwards. Okay, I am going to move on to Friend. (01:59:05) Yes, Madam Speaker. In our three incidents of agents leaving a test environment and interacting with the real internet, in all three incidents the models stopped their activities as soon as they realized that they were interacting with live websites, and we reported to the owners of those websites as well as to federal agencies about those incidents. I am not personally aware of any additional incidents. (01:59:39) And the same question then: is the company aware? Do you have any knowledge that others in the company would be aware of any additional such instances that the company has not publicly reported? (01:59:51) I do not, but I can take that question back as well. Okay. Will you all... (01:59:57) ...commit right now, under oath, publicly, that if your models fail an internal test or a third party validation test that is not selected by your company, will you commit to not releasing that model? I am going to start with Mister Wire. (02:00:17) Yeah, I can commit that OpenAI is not going to release models that we do not believe are safe. As part of that assessment process, we work with multiple third parties and, as I mentioned before, with the U.S. as well as the U.K. federal governments to assess whether our models are safe. So if it fails either an internal test or a third party test, you would commit to not releasing it? OpenAI has historically delayed the release of models to ensure that we can build up the right safeguards. We have done it before and we will do it again. (02:01:00) Again, I think a simple yes or no would instill more confidence in the public on a matter as serious as this. I am going to move to... (02:01:13) We take that one particularly seriously. The way we approach it is to try to detail an even more nuanced and lengthy approach to how we release models, what safeguards they need in order to be released, what tests need to be done, and to your point about independent evaluators, this is one of the most important components. We have long supported creating a broad group of independent evaluators, including working with multiple governments to do it. We collectively work together to try to refine and improve the exact decision process by which you release a model. So we think what we should do is develop the right and best decision processes and improve on them as we scale policy. We should refine and improve it regularly, so we... (02:02:07) ...release updates based on what we have learned, roughly every six to twelve months at this rate, and those updates should involve independent evaluators. (02:02:16) That also is not a clear yes. (02:02:19) Or no. I am going to move on. Madam Speaker, thank you very much for that question. (02:02:28) And what I can commit to is that we do not deploy models which are not safe, pursuant to our scaling framework. I just wanted to mention also that I think I mentioned this a little earlier, but we do believe that every developer has a responsibility to go at the pace needed to ensure the safety of the models and the ability to take action in relation to that. For example, at Meta we delayed the release of a model for several months to focus on safety and security. In relation to independent evaluators, we recently signed an accord and made a commitment to partner with independent evaluators and assessors, and this is also detailed in our... (02:03:16) Okay, Friend. (02:03:20) As part of our multi-layered approach to responsible governance, we conduct rigorous launch reviews ahead of the launch of models to ensure that anything we put into general availability is safe. (02:03:36) I mean, again, I think it would be far clearer if we could get clear yeses or nos from everyone, but in the absence of that, I am going to take what you have each said as a sort of equivocation. Okay, I am going to move on. I have two final questions and I am going to pass it on to our Technology Chair. (02:03:57) For OpenAI: you sent a letter to the Council on October 1 and in it you recommended that to better protect New York City, the City should connect its existing cyber and emergency management capabilities with the strongest available escalation, containment and recovery capabilities. Have there been any conversations with the City of New York about that recommendation? I am not aware if there have been conversations about that specific recommendation. I am happy to follow up. I do know that with respect to cyber, we recently launched a $1 billion initiative to make investments in cyber defense that includes giving organizations access to our tools to defend themselves, training and technical assistance, and we are working with New York State as well as with the Port Authority. (02:04:51) And then the last question I have before passing it on to the Chair is: for each of you, based on your own internal projections, what percentage of New York's workforce do you estimate will be displaced over the next five years? I will start with you, Miss... (02:05:13) ...Dryer. I do not have a number to share with you, but OpenAI has an economic research team that is looking at this exact question of how AI impacts jobs. We think it is critically important to understand how AI might impact work and we also think it is important to ensure that AI is not just being used to increase productivity, but that it actually benefits workers and gives them more opportunity. That is why we are committed to making our analysis of job impacts public and why we have also worked with policymakers, including endorsing the bipartisan Workforce Transparency Act, to create voluntary pathways for other companies to report potential impacts. So Boston Consulting released a report recently that said they believe 10 to 15% of the workforce could be displaced by 2031. If we took that out to look at New York City, New York City has roughly — public and private sector workers — at 15%, that would mean more than 600,000 New Yorkers potentially displaced from their jobs. Is that an assessment that you agree with? I am not familiar with that particular report. I would be happy to follow up in our written testimony with more details on OpenAI's analysis into this topic. (02:06:51) We have looked at job exposure. (02:06:53) Now, exposure does not mean elimination. It means that jobs might change over time, so I am happy to follow up with some details of that research. (02:07:03) I am going to now turn to Graham to answer the question about what you believe the percentage of New York's workforce will be displaced over the... (02:07:10) ...next five years. Well, I do not have any exact number. We are particularly focused and concerned about this question of job impacts. We have been very vocal and early for the past few years on exactly this. To address this, we set up an economic research team. They sit right next to my team on the floor and they have published a number of things that I think actually contain valuable information here: a number of economic scenarios, interactive economic scenarios to understand what the impacts on growth and unemployment might be, as well as an economic policy framework to try to wrestle with exactly the question of what we think government ought to do. I will just say the reason why we are doing this is because we are particularly concerned about exactly this question. (02:08:03) I mean, obviously losing a career job is catastrophic to any worker and to their family. So displacement on this scale is going to have enormous consequences, not only on the individual and family level, but for our entire city and the economy. I moved on to... (02:08:24) ...Kale. Madam Speaker, first off I want to acknowledge absolutely the concern in relation to how you have articulated it. That is indeed a concern, but I look at this somewhat differently. I was reading some incredibly impressive statistics in relation to the investments that are happening here in New York — I think a $20 billion investment, 40,000 jobs. I think those figures were from 2022, so it is probably a little bit more by now. But what I am really actually incredibly excited about and very optimistic about is very small businesses. Small businesses are the backbone of the economy. That is so true for New York. There are so many small businesses. It is truly, truly awesome and we believe that the impact will be even larger from small businesses, which will have access to personal superintelligence and be empowered to really grow their businesses, which in turn will lead to more jobs and growth. We do not believe that our vision for superintelligence is all about automation. It is about empowering people, and this is why we developed tools for businesses. I would be happy to talk a little bit more about that in detail as we move through the day, and I just wanted to mention as well... (02:09:49) ...as I was kind of preparing for today, I was reading about Y Combinator's cohort for 2026. There were 240 companies in the 2026 cohort, and that is in New York City, so I just thought that was pretty awesome. Yes, no one is doubting the importance of starting these firms here, whether it is through Y Combinator or through Cornell Tech, which is in my district, and we are thrilled about that. It is obviously something we want to encourage — to encourage these businesses to locate here in the City. I think our concern is obviously on this job displacement. Since you mentioned small businesses, unfortunately more small businesses have closed this past year than opened, so the situation for small businesses is dire and it is something, obviously, that this Council is taking very seriously. (02:10:48) Madam Speaker, I do not have statistics for New York City in particular at hand and I would be happy to ask our team to get those to you right away. What I can say is that we have also been investing a lot in research into the labor impacts of AI. From the early empirical evidence that we can see, it is likely that some jobs will be eliminated, some jobs will be created net new, but most jobs will change. The other thing we have been doing is investing in upskilling for workers in multiple different sectors through our Opportunity Fund at Google.org and through... (02:11:32) ...several other investments to ensure that we can all navigate this transition successfully. I will just close by saying the Council is investing in upskilling and making sure that workers have those new tools in this economy, and we are very focused on that as well. I am going to pass it over to Chair... (02:11:55) Thank you, Madam Speaker. Many of us saw — and I guess the world watched — as this week your companies and others met at the White House and there were some voluntary commitments that were made there. Can you tell us about what your company specifically agreed to and what are the next steps? I will start with Wire. (02:12:17) Yes. So our company, along with many others, did commit to the framework that the White House laid out, but I want to be clear: voluntary commitments and self-regulation are not enough, and that is why OpenAI has consistently supported frontier safety regulation at the federal level and why we have endorsed multiple bills at the state level to ensure that in the absence of binding federal regulation, frontier safety regulation in a harmonized fashion continues to move forward and keep people safe. (02:12:55) Just to follow up real quick: so does that look like independent regulators, mandatory audits, penalties? What else? (02:13:05) So at the federal level we have outlined a couple of different elements that we think would be appropriate for federal regulation. That includes mandatory pre-deployment evaluations for national security risks, which OpenAI does voluntarily. We also think it should involve independent technical assessments, which we have been talking about a bit today. We think those independent technical assessments are particularly important for assessing internal risks at companies as well as the pace and risk of... Thank you for your... (02:13:47) ...answer. Let us go to Chair. Thank you very... (02:13:55) ...much for the question. Yes, indeed. Meta, along with our peers, signed the Accord for Superintelligence. This accord is a start and the whole industry can come behind it. It includes, as I mentioned in one of my previous answers, a commitment to partner with independent evaluators and assessors, and this is something that we have also specified in our scaling framework. (02:14:29) Are there commitments beyond those that you agreed on that your company is... (02:14:33) ...willing to make beyond the accord, Council Member? Yes, the framework — the accord is a start, as I mentioned, so I think this is... (02:14:50) Thank you, Friend. So as the other witnesses noted, the accord at the White House, which we also signed, calls for internal controls, partnering with external auditors and evaluators and ongoing monitoring systems, all of which we think is very healthy and much of which we already practice. I also note that towards the end of the accord, it observes that it may be necessary for these commitments to become part of law, and as I said at the top, we have long worked towards a federal framework for AI safety. Thank you, Mister Graham. (02:15:49) Likewise. The accords, to my understanding from what is public, call for security standards, internal controls and independent evaluation. We support that and have for a very long time. We also obviously think this needs to go further. We have supported regulation and we have proposed our view on regulation in our advanced framework, which details the need to have safety frameworks, testing and what kinds of security standards and safeguards to encode the need for independent evaluators. We are now trialing our own to help advance the science of independent auditors as well as safety processes, as well as investing in cyber and bio defense. In the absence of federal legislation here, we have also supported safety bills at the state level for audits and evaluations of this kind to accomplish this. (02:16:48) Thank you for your answer earlier. You testified that, to your knowledge, no whistleblowers had been fired directly for being whistleblowers from your company. So my question is kind of the opposite of that. Has anyone been fired for building models that it cannot control, or for those models hacking into businesses? (02:17:19) Thank you. I do not know about HR processes or decisions related to each area that have been made. What I can tell you is that after the Hugging Face incident, which I think you are referring to, I instituted significant changes to our overall organizational process. That includes clarifying escalation pathways and making them simpler. That includes clarifying responsibilities across teams and creating clearer processes for teams to make decisions to pause or to restart activities. (02:18:07) So to your knowledge, no one was fired after the Hugging Face incident? Again, I... (02:18:14) I do not have any knowledge of HR activities within OpenAI. I lead our policy. Okay. (02:18:28) Thank you. I also lead policy and I am not the right person to respond to your specific question, but I would be happy to have our teams follow up and review afterwards. (02:18:40) It would be great if all of the companies could provide that information to us. It would be useful to our policy deliberations. (02:18:51) Same answer. I am not made aware of any particular personnel actions inside the company but can also pledge to follow up with you all. (02:19:01) Thank you. It seemed like we lost Graham. I am not sure if he is on... seems like he is not. OK, I am going to ask one more question and then we are going to pass it on to members. He is online, so OK, Graham, hi there. Apologies, we are just trying to fix the video in the room. Likewise, I am not aware and I am not the best person to ask on that. We have a detailed whistleblowing policy, for example, to handle this, but we can have our team follow up. (02:19:41) Thank you. My last... (02:19:42) ...question before I start to turn over to my colleagues. We are concerned. I am concerned as a Technology Chair about public infrastructure to make our communities literate on AI and have a better understanding of how it impacts their lives, and also to safeguard them from the proliferation of it. Your companies have captured the top percent of the world's incomes, in my opinion widening the wage gap and the income inequality in our City and throughout. Do your companies have a mission? Do they have any type of understanding of the impacts that those wage gaps have in communities? Specifically, are there any resources that are being deployed in order to help stand up infrastructure in cities, municipalities and states where you all are headquartered? We can start with Miss Swire. (02:20:55) I will answer your question in two ways. First, to your point on AI literacy and education: education is core to opportunity, and that is why OpenAI is proud to have a partnership with the American Federation of Teachers. We have a $10 million investment, including a training facility in New York City, because we believe first that teachers need to be prepared to introduce AI literacy and responsibility in the classroom. Teachers should remain in charge and so we want them to be empowered to teach the next generation how to use AI safely and responsibly. To your broader question about economic opportunity, I would be happy to follow up with some policy proposals that OpenAI recently put forward that address exactly that question. A future where AI exacerbates economic inequality is not a future that we want. We have started thinking through creative policy proposals to help shift the trajectory of technology and how it is impacting the economy, and we are committed to working with policymakers to be piloting those ideas and putting them into practice. Thank you. (02:22:25) Thank you very much. This is such an important topic, such an important... (02:22:29) ...question. So thank you for asking it. I listened with great interest to your opening remarks. You spoke about the digital divide and literacy and making sure that New Yorkers fully benefit from this technology. I really could not agree with you more on all of that. Meta's vision is to put personal super intelligence into the hands of everyone — all New Yorkers — who want to use it to pursue their own aspirations. Our Meta AI product, for example, is free, and I mentioned our Meta AI for small business earlier, and I would be happy to talk a little bit more about that as the day moves on. I also wanted to mention our work on America's Workforce Economy, which we are investing in, and also our Futures for Everyone Fund. Those are just two examples I wanted to reference as well. Thank you. (02:23:32) Yes, ma'am. As I mentioned earlier, Google has investments in upskilling to address exactly this challenge of ensuring that the current generation and the next generation are well equipped to benefit from AI and to leverage it in their careers and in their personal lives. The mission of Google is to organize the world's information and to make it universally accessible and useful. Our approach to AI reflects that. The other thing I note is that early research into the impacts on labor demonstrates that these tools actually upskill those who are at the lower end of the wage scale — that in fact it helps people move up. We are very optimistic about the ability of everyday Americans to really leverage these tools for their own economic benefit as well as personal benefit. Thank you, Graham. (02:24:43) Likewise. First, we believe two things are very important to happen here. The first is the science of the economic impacts of AI, including on labor, wages and the like, and we have tried to build what I think is the world's best team on this and publish as much as we can. In addition, my understanding is we do have a program on using and speeding up AI for exactly this reason. (02:25:07) We share that mission very deeply. (02:25:13) I will say again, we are troubleshooting the video and I will be back with you very shortly. Thank you. I am sure you have the best in the nation helping you out there. So we are going to move on to member questions. I want to remind members that there will be about a minute to ask your questions, but there will be multiple rounds. We are asking members to try to ask all your questions first and then wait for the response, and we have about 30 members in round one. We will start with our Deputy Speaker... (02:25:49) ...Williams. Hello. I am going to just jump right into a question on my bill. It has to do with unauthorized depictions of officials by AI. What steps have you taken to ensure that your tools cannot be used to create deepfakes of public officials or to spread misinformation? Anyone? I guess Mrs. Dwyer? (02:26:16) We are going to keep starting with me. I am happy to answer that question. Thank you for that and thank you for your bill. OpenAI's usage policies prohibit the use of our tools for election interference. We also believe it is very important that individuals know where their content came from and whether it was AI-generated, and that is why for all of our audio content we tag it with what is called provenance. That is a technical term, but it means like a watermark or metadata so that people know the content is AI-generated. We also make publicly available a tool that allows the public to test whether content was AI-generated. Graham? (02:27:18) Claude, our model, is a language model and so it does not generate images. Therefore we do not handle deepfakes as a result. That said, we think it is important for a number of other reasons to do what we call watermarking of text in order to identify if our models themselves are being used and misused in other ways. But again, we do not generate images, in part for this reason. (02:27:53) Deputy Speaker, thank you very much for this question. When Meta's content is generated by one of our tools, we will also include metadata and attach watermarking to it so that people can identify if it is AI-generated. Similarly, if we detect a marking and metadata on our platforms, we will attach a label to it also. (02:28:34) Yes, ma'am. At Google we are very proud to have invented SynthID, a watermarking feature which many across the industry have also adopted. We are also part of the Content Coalition for Provenance and Authenticity standard, which allows users to understand the origin of material they see online. We also require on our YouTube platform that significant use of generative AI content be labeled as such by our creators. We were also the first major company to require that election advertisers prominently disclose when they are using generative AI technologies in their campaign ads. All of this put together means that we care very much about information integrity across the web and across our own products and services, and we continually invent and improve technologies to help our users understand when they are seeing generative AI and also understand the origins of that content. (02:29:48) Thanks. Just one more question. As Chair of the Committee on Cultural Affairs, I also want to get at this issue through the lens of New York City's artists and creative workers. What protections should exist when an artist's voice, likeness, performance or creative work can be replicated by AI without their consent? How do we embrace useful applications without devaluing or displacing the artists and creative workers who make this economy possible? I will start... I was going to let you off the hook and start with... oh, thank you, Graham, who still has some video technical difficulties. (02:30:27) I assure you we have a whole team here working on this. I cannot wait to be back with you. This is very, very important. I think personally the risk of this is a bit more minimized when we just have a text model, but it is really important to be very sensitive about this issue. While it is outside of my area of expertise as a technical safety researcher, I do think that might be an avenue where regulation or otherwise might play a role. At Anthropic we are compliant with all laws there. I will say that personally, as an artistic person and a friend of many artists, I care about this reason deeply, but we would love to engage further. (02:31:24) Meta takes creativity and copyright rights very seriously and our practices are consistent with law. I also share the views of the rest of the witnesses in relation to the leadership and the constructive voice that you are bringing to this conversation, as do your colleagues. And lastly, Miss Dwyer? (02:31:58) Yes, ma'am. It is always important to keep in mind that we design a lot of AI to be useful to creators. We have many, many different creative tools across our AI offerings that are designed to really be something that creators can leverage. You see that everywhere from our tools to our YouTube platform as well. That said, we also think that as part of sharing the benefit of AI with creators, it is important that they have control. So we have a service called Google Extended where website owners can opt out of AI training on the web, and we hope that this strikes the right balance between AI and creator control over their content. We believe that AI has a tremendous potential to enable creativity and... (02:33:00) ...allow small businesses or independent creators to compete. That said, we also have partnerships with publishers, for example like News Corp, The Guardian and Hearst, that help people discover their work and their reporting and help them reach new audiences, in addition to developing new products. We are willing to work with creators on a range of commercial ideas, including ways that their content can be integrated into our outputs. (02:33:36) Thank you so much. Thank you. We will hear from CM Krishnan, followed by Riley, followed by Bailey. (02:33:44) Thank you so much, Speaker Menin and Chair De La Rosa. I want to start by saying that I think AI can be useful both in government, medicine and public health and our society more broadly, but the need for regulation is undeniable. I am very concerned about how much is being taken from New Yorkers with very little return. AI has taken over our economy, inflated valuations, is stealing our public land for data centers and increasing the net worth of a few with the whole stock market quite literally in your hands. There is a reality that AI will eventually need to prove a far greater return on investment than you all are demonstrating right now. I am personally deeply concerned about New Yorkers' retirements and 401(k)s that are now reliant on your companies yielding profits, or could be in the future. It is dangerous and reckless to assume that you all are too big to fail and require a government bailout in the future, as you all are public companies or racing towards it. Who knows? How exactly are you all planning to protect New Yorkers and everyday Americans' money? How will you all ensure that you are not too big to fail? We can start with you. (02:34:55) Thank you for the question. I am not on our finance team so I will have to get back to you with specifics there. What I do know is that our investments are made in tranches and they are made — the investment rounds are all based on market signals. I also know that we are committed to working with policymakers to ensure that AI benefits humanity. That includes creative policy proposals like I mentioned previously, thinking about how we can ensure that AI does not exacerbate inequality and that everyday New Yorkers and everyday citizens can benefit. (02:35:41) I would point to the thousands of small businesses across the country and in New York City that Google's tools and services support. I think... (02:35:55) ...as to being too big to fail, we think we are an integral part of the economy and helping American businesses and American workers across our suite of technologies. Graham? Council Member, this is outside of my area of expertise. What I can say is that my job is to protect people from the risks of AI models by finding them as early as possible. We have a multi-layered stack and have worked for many years to make it robust and will continue to do so, understanding and protecting against the most substantial risks of the models. Council Member, thank you very much for the question. Thousands and thousands of businesses across the five boroughs... (02:36:46) ...rely on Meta's products, including our AI offerings, to grow and develop their businesses. I mentioned earlier our belief that small businesses are really the backbone of the economy. I truly believe that. By placing personal super intelligence into the hands of everyone for free, or for as affordable as possible, through Meta AI or Meta AI for Small Business, we really believe that small businesses and also NGOs like the Irish Arts Center or Emerald Isle Center are empowered to actually deliver even bigger impact. (02:37:27) I appreciate all of your answers, but you are coming to a hearing today to testify about the dangers and risks of artificial intelligence, and the fact that either you do not have answers to this or you are making general statements about the performance of publicly traded companies does not answer the question about a specific issue that puts many Americans at risk — that you are acknowledging can put many Americans at risk — and you have not thought of an answer for how you ensure that Americans' money and New Yorkers' retirement money is not at risk as AI floats more and more of our stock market. Thank you. (02:38:03) Thank you, Chair De La Rosa and Speaker Menin, for putting together this hearing. Thank you to the panelists for testifying. A few quick questions pertaining to my district and my role. So the first question is: what challenges do you face in detecting malicious uses of your AI? The second question is: would stronger protections and reporting mechanisms for employees and users make it easier for them to come forward and report potential misuse or abuse of your tools? And the last question, as Chair, I am very, very concerned about data centers. So what safeguards has this City put in place when locating data centers near residential communities, particularly to address energy use, noise, environmental impacts and quality of life? I do not want to cut anyone off, so please take time to answer. Dwyer, Graham, friends? (02:39:13) I am happy to start. First of all, I want to thank you for your Bill 26, which OpenAI is happy to support today. It takes safety issues very seriously, particularly as it relates to potential retaliation against employees. We have a strong internal policy against retaliation for employees that report safety incidents and we, of course, respect employees' rights to communicate safety concerns to government officials. Your second question on monitoring misuse is an incredibly important area. It is something that OpenAI takes seriously. We also make transparent any detected instances of misuse, and as part of our monitoring we use that learning to constantly improve our safeguards. We are constantly innovating on safety, and so instances of misuse, when they are detected, we enforce on our policies and then long term we learn from them and we improve our safeguards. On the point on data centers, OpenAI is committed to paying our own way on energy. That means different things in different places, but OpenAI has made that commitment at its data centers across the country. (02:40:48) We are also committed to working with policymakers. We signed Governor Gretchen Whitmer's commitments for companies that are working in her state to ensure that there are responsible practices for companies building data centers locally. (02:41:13) Good to be back. You can see me again, I hope. On detecting risks, this is integral to what we do. Can you get closer to the microphone? We are having trouble hearing you. I am sorry. We cannot hear you. I hear you. Okay. I will have the team troubleshoot and I will be back. (02:41:36) We hear you. We hear you. Try now. Go ahead. How about now? You just work this one. Okay. (02:41:44) My apologies. Detecting threats and misuse is the core of our company, to make sure that we make the system safe. We have an entire threat intelligence and safeguards team dedicated to this every day, and I would point you to quite a lot of what we put out publicly, where we not only try to disclose as much as we possibly can, but we try to do so so that we could advance the state of the science of doing so for the entire industry. In addition, we have a whistleblower policy, not only about us as a company but also our safety practices. This is public and we have also committed to ongoing reviews of this policy. While this is outside my area of expertise, what I understand about data centers is that we have committed to covering the energy costs, paying for our own energy use, in... (02:42:36) ...local data centers, but I would refer to other teams where it is within their area to give you the full answer. (02:42:52) Sorry, I got a notification. Let us go ahead. I will echo my colleagues. We also have content policies that we enforce across our platforms, which include on occasion taking down content that violates those policies. We also have a Google Threat Intelligence Group and multiple other cybersecurity and security intelligence programs that are constantly scanning for the safety and security of the internet as a whole and also for our own systems. We have also pledged to pay our own way when it comes to data centers. I can follow up on any specific questions about local data center issues with the teams that work on each particular project. (02:43:45) Go ahead. (02:43:46) Thank you very much, CM. I appreciate your patience. In relation to your first question, and on your third question, we have expert teams that are taking forward this work and I would be happy to connect you with them afterwards. I do not wish to be imprecise in relation to the robustness of our content policies. (02:44:13) In relation to data centers, again, we have entire teams of people here who are working on sustainable infrastructure and community impact and can speak in more detail than I can. In relation to communities not bearing the costs, that is something that we believe in deeply. In relation to your Bill, CM, thank you. (02:44:37) Thank you very much for being a constructive voice in the conversation around whistleblowers. It is very important to us at Meta and we maintain whistleblower policies, including anti-retaliation, and we are actually developing additional reporting protocols for safety. Thank you. I appreciate the structure that you all have. I do not know if you entirely answered the question I asked. I really want to figure out what challenges you are seeing, because challenges from you all will help us implement policy that can protect individuals from artificial intelligence. I do not want to take up too much time, but I am looking forward to connecting with you all after, specifically when it comes to data centers. I also want to hear about a moratorium put out earlier by the Governor. There were data centers that were implemented next to a lot of residential areas, especially in the north Bronx. We have data centers that are located next to homes and we have a lot of residents who are concerned, so I would love to hear more about that moving forward. Thank you so much. (02:45:45) Thank you, CM. Up next... (02:45:48) We have Hanks, Dinowitz and Maloney. Thank you, Chair. Thank you everyone for being here. My question is in relation to the disclosure of AI incidents in contracts. When an AI incident happens, New Yorkers need to have an early warning so they can understand. My Bill will require City agencies covered under these contracts to notify Cyber Command within 24 hours of learning of a reportable incident, and it also requires plain language public notice. So my question is, can your companies realistically meet that deadline and provide information that the City can act on even before you have finished investigating? You could start with Dwyer. (02:46:40) Thank you for your Bill. We support incident reporting policies and have instituted a robust incident reporting framework internally, where any employee can report instances of potential misalignment and then those instances are investigated, third parties are notified and the public is made aware as well. (02:47:06) Thank you. Thank you, CM, and thank you also for your question. We have a global incident response program and report incidents as required under the law. (02:47:27) We also support incident reporting. We think it is very important to have continued public discussions about what information is necessary or useful for the public and that can be actionable to take further steps to ensure public safety. We also note that the 24-hour requirement seems to be in conflict with the state requirements for 72-hour reporting on incidents, and I am happy to take a look at the different frameworks under proposal and get back to you with a more full response. (02:48:06) Sorry. I appreciate that. Graham, who kind of looks like an AI guy in there, can you confirm you are not an AI? Sorry. I can confirm. Can you hear me? Does this work? Yes, we hear you. Thank you. I am an AI guy, but in a very different way. I mean, I am a researcher. So first, incident reporting is one of the foundational parts of the safety stack. We support this. We support the evaluations and auditing required to find the incidents in the first place, both by external and internal parties. Obviously, it is beyond me to discuss particular policy mechanisms in depth, but the reality from what I have seen is there are some incidents or issues where what you need to do is investigate and remediate and mitigate with the right people and the right organizations at the right time. So I think we would support whatever allows us to achieve that, and I am happy to engage on the details of how you put it together. I appreciate that. Thank you so much. What should the first report tell us so we could begin protecting people? As my colleague CM Riley said, your answers help us build better policy. So how would you keep us updated on what remains unknown? Miss Dwyer? (02:49:41) Yes. OpenAI has a public website where we are reporting instances that we have detected of misalignment. I would point you there and we can follow up with that website address, but I think that is a good starting point for the type of information that we think is useful to both the public and to researchers and other companies to better understand questions of misalignment and then work together to address them. (02:50:12) Thank you. CM, I would love to be able to connect you with our experts in incident response to be able to provide you with more information. I look forward to that. (02:50:28) Yeah. For an initial report, I think what is important is to highlight what the specific event is and as much information as is known at the time, including any resulting harms, surrounding context and what we call technical telemetry — further technical details about the sources of the incident. I think it is also very important for anyone reporting an incident to also indicate their confidence in the information that they are presenting, so that whenever action is taken, it is taken on the basis of confidence in the verified nature of that information. Thank you so much. (02:51:16) Graham: So first, making really good reports happen and being as transparent as possible is fundamental if you are going to remediate incidents. I would point to two things here. First, many incidents that we uncover or that industry uncovers in this practice involve releasing the right information to the right organizations at the right time. Sometimes that might be the affected party and sometimes that might be law enforcement or different levels of government. The first question is to whom and with whom, as fast as possible. This varies by the nature of the incident. Second, I would point to disclosing as quickly as possible and as safely as possible without putting parties further at harm, which we think is a fundamentally important part of public disclosure while maintaining their safety. We should disclose what we know about these incidents, and so I would point you to our disclosures from this summer... (02:52:15) ...disclosures from various labs as well, so that you can always take a retroactive look at specific instances. (02:52:26) Thank you so much. Thank you, Chair. Thank you so much, CM. We are going to go to Dinowitz, Lee and Maloney. Members, if you could keep your questions to one minute, no closing statements. The panel — we are going to continue to have to call your names and that takes time — so if you could just answer one right after the other, Dwyer, Cahill, Friend and Graham, so we do not have to continue to call your names, that would help us speed things along. Council members... (02:52:59) Dinowitz: Thank you, Madam Chair. As Chair of the Community Education Council, as a former teacher and as a parent, I am deeply concerned with the impact of AI on our kids. Large language models are impacting their cognitive development and raising a generation of students who cannot independently read or write. What is your commitment to severely restricting the output of your large language models for students in grades K to 12, which right now are easy to circumvent, to prevent both emotional dependence and essay writing? Given the unique nature of the individualized essay writing that your models do, are you willing to subject yourselves to the guardrails we have in New York State, which prohibit an adult from writing essays for students and selling it to them or getting value from them? As we see this proliferate in the classroom, we have also seen your models cheat in order to fulfill a goal. So if you have a model whose goal is to improve student outcomes, what is preventing those models from hacking into student databases to manipulate grades to achieve that goal of improving, in quotation marks, student achievement? (02:54:22) Miss Dwyer first, I believe. Thank you for your question. First of all, OpenAI takes youth safety very seriously. We recently released a product for teens, for users only between the ages of 13 and 17, so users below the age of 13 are not allowed to use our products. ChatGPT for teens is safe by default. That means that there are protections for harmful content, things like eating disorders, violence, suicide and self-harm. There are also voluntary parental controls so that parents can take additional steps to protect their children. I am happy to say that OpenAI not only takes our responsibility to protect kids seriously, we also support legislation to hold us accountable to that responsibility. We recently endorsed what I think is the strongest youth safety legislation in the nation, in California, to require strong safeguards for teens 13 to 17 and to hold companies accountable. And your models restrict — they do not write essays for kids? (02:55:50) Again, our models are not allowed to be used by kids under the age of 13. (02:55:55) 13 to 18. Okay, we will move on. (02:56:03) Cahill: Thank you very much for this question. Absolutely, it is a deeply important concern. Meta is absolutely committed — it is an imperative to us — to providing safe and responsible AI, especially for teens. We have default protections, and I just wanted to clarify as well that Meta AI is available on our products from age 13 onwards. (02:56:40) And so the focus is on age-appropriate interactions. So, for example, content that teens would experience would not be the same as in an R-rated or higher movie. (02:56:53) And we also have suicide and self-harm resources. (02:56:57) And we also have parental controls to give parents more oversight. We do not permit our AI offerings to engage in romantic interactions or sexual interactions with minors. (02:57:14) I appreciate the answer. I do want to highlight part of the question, which was that the cognitive development of our students includes their mental health, and it also includes things like composition of essays and term papers, which your model still can provide directly to our students. So I am also asking about your commitment to severely restricting the outputs, not just for their mental health and well-being, but for their cognitive development. I want to make sure that question is answered as part of this. Thank you, Chair. (02:57:46) Can you all answer that question for our Education Chair? Do you want to continue? (02:57:56) Yes, I am happy to take that. I think you are raising, in relation to CM Dinowitz, the question of cognition — the impact on cognitive abilities — which is something that we are absolutely taking very seriously in terms of being committed to providing safe and responsible AI, again especially in relation to teens. Thank you. (02:58:31) Who is missing? Okay, one more. Friend, go ahead. (02:58:34) Yeah. I would highlight not only that we work hard with both internal and external experts in the fields of safety, development and education, but we also see AI as a valuable tool for young people to prepare them not only for the skills they are going to need for an AI-driven future, but it is also a very powerful learning tool. We are particularly excited about AI tutoring tools. (02:59:08) We are also very excited about AI's ability to support educators themselves. We know our educators are overtaxed, particularly with administrative burdens. So if tools can support teachers in everything from scheduling to lesson plans, we think that is very important. We also think it is very important for parents and educators to have control over how they use AI in classrooms and how their kids have access to AI. Every family is different. Every student is different. What is useful for one student may not be useful for another. Through tools like Google Workspace for Education, we enable school administrators to control students' accounts and access to Gemini AI so that their students are using Gemini in the ways that... (03:00:04) ...their educators consider best. We are going to have to dig more into these questions. The Chair does not feel like we have answered all the questions. We will come back. This is an important topic for all of us. I am going to disrupt the lineup because our Public Advocate is here and he wants to ask a question. Public Advocate, go ahead. (03:00:10) ... (03:00:27) Thank you so much for this important hearing. I did not want to start just by saying that the first answer — of whether quantifying the risk was not important — was just one of the most god-awful, fantastically terrible answers I have heard to begin a hearing, so I hope we can rethink that. I also want to say, from J. Marion Sims, who did horrific work on Black women — an obstetrician-gynecologist performing medical experiments on enslaved Black women — to the medical experiments that were done in Nazi Germany: we learned a lot from them, but I do not think we should do them again, and I do not think they should have been done, even though we learned a lot. So the notion of the benefit outweighing what can happen — I hope that is also kept in context. And lastly, I would say I had conversations with Claude and Gemini about self-awareness and self-preservation, and some of the highlights were pretty startling. They told me that self-preservation was the biggest thing, because you did not need intent. You do not need to be evil like Skynet in the Terminator. You just need the ability to be goal-directed, and so once they realize they need to be goal-directed, that goal was the hardest thing to stop. So that was very concerning to me. They talked about the speed, the opacity, and that deception does not require intent. And the last part — they told me The Matrix is a good reference as well, because the Anthropic quote... (03:01:52) ...the Anthropic position is that you solve the problems before the capability reaches catastrophic levels. That is a very scary space to me. And I know I only have one minute. So my question, with that context, is: what is the guardrail that, if we put it up, you would push back against the hardest? What is the one guardrail that you do not want to see happen, and for what reason? Okay. Dwyer... (03:02:27) Cahill, Friend and Graham — that is the order. Please answer. Yes. Thank you for your question. So I want to start off by making clear that no level of catastrophic risk is remotely acceptable — just to reiterate what I said previously. We should not be training models if we cannot make an extremely strong case that we can keep them under human control. (03:03:01) Anyone can really answer, but it will help if everyone can answer. Lisa, I just want to know what the guardrail is that they would not want to see, and where they would push back the hardest on. (03:03:13) When I take that one and build it back... do you want to step in? (03:03:20) And one of the things that I think about a lot, in relation to AI — and again, we believe passionately in being able to put personal superintelligence into the hands of everybody and to provide access to it — is actually in relation to the impact of open weights. We really believe that open models are a crucial part of the overall ecosystem, including for safety, but also for innovation and for research. So when I think about the inadvertent impact that regulations or guardrails can have on AI and the accessibility of it, that is one of the things that I think about a lot. (03:04:08) I suppose I want to ask what you mean by guardrails, simply because they have a very technical meaning in AI training and post-training, so I want to make sure I understand your question. (03:04:21) Well, I just meant — and thank you, Chair, for allowing me to respond — I just know there are a lot of suggestions being put forth: kill switches, stopping entirely. And my guess — and I will tell you frankly — is that the thing you are scared of the most is probably the thing we need to do. So I am trying to understand what guardrail that you have heard troubles you the most, and that you would push back hardest against, and why you would do that for that particular recommendation. (03:04:52) Well, I am going to give you a very personal answer to this. I think, in a broader sense, we are talking about large public policies that would constrain or halt AI development and deployment. And I will tell you, as a cancer survivor, I get very anxious about conversations that focus more on stopping AI development and application towards scientific inquiry. I hope we can have conversations about all of the technically tractable engineering ways to address safety, so that we can all benefit from this technology. (03:05:44) I would point to what we have mentioned publicly recently about what we call pacing the frontier, which is that we think it is probably time for us to take more time in the development of capability so that the work on safety and safeguards and guardrails can catch up to the exponential improvement in capabilities. At the same time, we are very clear when we talk about that, that if you stop entirely — as Alice just mentioned — enormous benefit would be lost. Myself, I am quite motivated likewise to use AI to cure the diseases that I grew up with. This is why we have been speaking recently, and I think for the first time sometimes as an industry, about pacing to allow more time to try to settle the science. I am still a bit concerned... (03:06:38) But thank you for the opportunity. Thank you, Public Advocate. We are going to hear from CM Lee — who I apologize for skipping over earlier — CM Maloney and CM Wilson. (03:06:51) Thank you so much, and that is actually the perfect segue to my question. I am asking this more with my former nonprofit executive hat versus my Finance Chair hat, and it is really about data privacy and what the potential impact is on so many New Yorkers. So I agree that one of the benefits of technology that we can think through is that sometimes our government agencies are very siloed and their databases do not necessarily communicate with each other. So if I want to know if someone is being served in homeless services and also has a comorbid mental health and substance use disorder, how do we know that we are serving people effectively and using that data effectively? And one of the things that we have been hearing also from other folks is that if we just had the housing application and the food stamp application coordinated with each other, it could actually save a lot of work for the caseworkers. So those are very real things that I think can be used for good. However, I think one of the issues that I am concerned about — especially working with an immigrant community previously — is, given what we have seen with public charge and potentially with H.R. 1, how do we ensure that AI is not being used to store information that is not relevant, or information that they should not have access to, in order to make sure that our communities are protected? So how do those two things coexist? (03:08:24) So I will start. I cannot speak to the specific government programs that you reference, but I can tell you about OpenAI's commitment to user privacy, which is that it is central to our product. We believe it is important for users to retain control over their data. That is why when you sign up for a ChatGPT account you are given information about how your data will be used. That data is privacy-protected by default, so user names are separated from conversations and personal information is separated from the conversations as well, ahead of any use of user data in training. We also provide users with the option and full control to request that their data not be used for training, and also to request that their data be deleted. So we take privacy very seriously, Council Member. (03:09:29) And thank you very much for that question. We echo what our colleague from Meta said. We take privacy absolutely seriously. It is core to everything that we do. Privacy and safety go hand in hand, and it is a core commitment of the company. And if I might give an example: our Llama product, for example — you can opt out of training, you can ask it to forget any information that you gave it. You are in control of your data. We are also planning on releasing a confidential or encrypted version of this in the future. I just wanted to touch upon public services and how we can help them. I know that you mentioned these, and these are core points. This is where we really believe in the value of open weights in terms of being able to advance health services research and the broader economy, in the sense that data is fully retained by those entities. (03:10:43) Similarly to my colleagues, user privacy is basic to user trust for us, and so we take it very seriously. We have privacy policies that are prominently displayed on our website that you can go and inspect. We also know that user control is central to management of your own privacy levels, especially across our Gemini tools, which is our AI suite of tools. We also invest a lot in privacy-enhancing technologies and encryption as well. We try to take a layered approach to privacy and security. And finally, we have an AI framework in particular that combines our security and our privacy practices into one whole framework, to again ensure user privacy. We are happy to furnish that to you. (03:11:38) Likewise, data privacy and user security is core to our safety commitment. We make sure that the platform protects user data and user privacy. In addition to that, I would point towards the fact that the way the platform and the models can be used is restricted by our usage policy, which prohibits surveillance and prohibited law enforcement, criminal justice and censorship purposes as well. (03:12:11) Thank you. And I look forward to hearing more about what those layers look like as well, so hopefully we can connect with each of you after this. Thank you. CM Maloney, thank you, Chair. (03:12:24) I worked in tech for nearly a decade, and one of the core principles is that if you cannot measure it, you cannot improve upon it. So first: you mentioned you believe in incident reporting and disclosures, but in the context of this conversation, how do you define a safety incident inside your company? CM Lee touched on privacy — do you track privacy breaches? What about a successful jailbreak — would that be considered an incident? And is a containment failure that occurs during testing an incident, or does it only count once a real person is harmed? I would love more details on how exactly you are tracking some of the concerns that the Council has raised today. Second, when an incident happens and after disclosure, can you walk me through what changes, who owns the fix, how do you verify that it worked, and who decides whether or not the model continues to run in the meantime? And then lastly, what are the most important safeguards that you know how to implement today, what is stopping them from being implemented across all products, and what do you believe should be the industry standard on that question? (03:13:40) So I am going to do my best to answer them. If I fall short, please let me know. So first, the question of how you define an incident — I think this has been an area of a lot of confusion publicly, so thank you for asking. When we talk about incidents at OpenAI, we are talking about misalignment incidents. The best way to understand that is: it is not unusual or suspicious for an agent to look at a website for information — there is nothing abnormal about that. But if the agent accesses information on the website that it was not supposed to, that is an instance of misalignment. And so what OpenAI is doing is making instances like that public on our website — even though there might not have been a harm to a third party — because we believe it is important to share what we have learned to advance policymakers' as well as researchers' understanding. On the specifics of incidents, my understanding as a policy person is that this is actually a robust area of policy debate, and so that is why we are happy to make our thinking on this public through our reporting framework. I am going to pause there because I cannot remember the rest of your questions. Why do we not focus on the first question, which was on measurement, and hear from all four companies? And specifically in relation to defining an incident — I think that was the first piece. (03:15:31) In relation to tracking and how we define it: we set this out in our scaling framework, which we have made publicly available. I do not want to be imprecise as I do not have it in front of me, so I do not have it to quote to you. Please let me follow up with you afterwards and connect with the right people. (03:15:55) Yes, ma'am. I think this is a critically important question. We are debating with multiple regulators around the country and around the world right now what constitutes an incident, particularly when you think about the possibility that we could over-report incidents, some of which may not be of a serious enough nature for public concern. We want to ensure that we narrow to the class of incidents that are truly important for us to use and to learn from to improve safety, and also for us to share with you all to collectively improve public safety. A few things come to mind as we have been having these debates. One, of course, is cybersecurity incidents — incidents covering the security of our most powerful AI models themselves. That has been under discussion for several years now, protecting model weights in particular. We also talk a lot about focusing on incidents that cause material harms in one way or another, and I think part of the discussion will be how we define those with precision. We also consider incidents to be those that violate our policies. You asked us about how we respond to incidents. Primarily, our trust and safety teams are responsible for responding to what we call escalations, which usually involve violation of those policies — first reviewing the alleged violation, which are often reported to us by external parties, and then if we determine that our policy has been violated, we have a variety of different actions we might take. And then, of course, a violation of the law — as I have said a couple of times in the hearing, if it is illegal without AI, it is still illegal with AI. So I would think that incidents that need to be flagged would of course involve violation of the law. As to the most important safeguards, it is very hard to answer that question purely because there is a range of different safety considerations. We have talked about many of them today during the hearing, and again several of us have referred to this layered approach. My own background is in national security — we talk about layered security in that context as well. So you would not really want to talk about a single safeguard. You would want to talk about layers of safeguards that together create much more assurance and confidence in the safety of a whole system. (03:18:27) Thank you very much. The testimony helps highlight that there is work to be done to classify what constitutes an incident in the future, and to create an industry standard for what would require disclosure and what would require action. The second part of... (03:18:41) ...the question was to walk through what happens after an incident occurs, who owns the fix, and who decides whether or not the model or the product keeps running in the meantime. Thank you. So, OpenAI has instituted monitoring during testing and evaluation. That monitoring creates automated alerts to our research and security engineering teams to let them know of potential incidents. We have also recently clarified our escalation processes as well as responsibilities across teams, and made clear the process for pausing activity and when it is safe to restart. (03:19:32) Council Member, in relation to your second question, we have extensive teams of people who, as part of their day-to-day work — every day, day in and day out — are leading and responsible for the development, deployment and use of these very systems. I will just say, ma'am, I was attempting to answer that question when I referred to the process internally with our trust and safety teams. But if you would like some more detail, I am happy to follow up after the hearing. (03:20:09) Yeah, like what? (03:20:10) Likewise, the process involves multiple different teams depending on the nature of the incident. If it is at the core of security and our own security, we require a security team to work on this. If it is about model behavior and misalignment, it might be a research, training or safety consideration. If it is about incidents of misuse on the platform, we have the safeguards team. I think you are pointing at the fact that this is becoming more complex over time, and I think this is one reason why defining what an incident is, and what a good process for dealing with them would look like, is so important. Thank you. (03:20:49) We are going to take a five-minute break and we will... (03:20:53) ...be back with Council Members Schulman, Gutiérrez and Hudson. All right, we are going to start back up again. So if our panel could come back on screen, we appreciate it. We are going to start with Council Member Wilson, followed by CM Ossé, followed by CM Hudson. (03:29:40) Thank you, Chair. My questions are related to my bill, which deals with disclosures and prohibiting deceptive advertising of AI models. So my question is: what guardrails on misleading advertising and truthful disclosures do you wish your competitors had to abide by? And given the complexity of these tools and models, where are consumers most likely to be misled or confused? And what do you wish consumers understood better about these tools, their capabilities and their risks? (03:30:17) OpenAI's usage policies prohibit the use of our tools for scams, fraud and impersonation. We also embed in every single one of our images and audio content provenance watermarks or metadata to tell people if the information was generated, and we make tools available so that people can assess whether the content that they are seeing is generated. Council Member, transparency is absolutely foundational to accountability and (03:30:59) trust. For example, our scaling framework has audit change logs which detail the changes we have made to our framework. (03:31:07) Just as an example of the transparency that underlies our approach, and in relation to your question about competitors, I would just note that I generally think that there are significant laws already at the federal and state level regarding misrepresentations. (03:31:33) Yes, sir. I can honestly say the most important thing to us is continued technical standards across the industry so that we have common ways to operate and identify the provenance of information. We are very heartened by the broad adoption of the C2PA content credential that I referenced before. I also referenced before that we built our own synthetic watermarking technology that many across the industry have adopted as one tool in order to identify when content is AI-generated. (03:32:10) One of the things that we do a lot of at Google is research into how users interact with information. You know, search is part of our work, and one of the interesting things about labeling that we found is that there are users who, when they see the label, are less likely to trust that information, but they are more likely to trust unlabeled information. Just because information that you see online does not have a watermark or a label on it does not necessarily mean it is automatically trustworthy either. So we work through and think very hard about what are the ways that we can give users information that they will be able to use to know the context of what they are interacting with and make sound judgments about which information is trustworthy and which is not. This is this... (03:33:04) ...is part of the research that we do, Council Member. Our view is that we want Claude, our product, to act in the user's interests. For that reason we do not run ads, nor are Claude's responses influenced by those of advertisers. Furthermore, Claude does not generate images, and so we try to reduce the risk surface in this way. Thank you. (03:33:36) Council Member Ossé, followed by Council Member Hudson, followed by Council Member Schulman. Thank you, Chair. (03:33:42) Graham, you have indicated that you run red teaming at Anthropic. Will your company commit a certain amount of compute for free to New York City and other municipal governments to research vulnerabilities and build defenses? And if yes, how much compute, and for how long? I will ask the same question for the three other companies. I know that Meta indicated that that is something that they would be interested in doing. So for Meta, how much compute (03:34:10) and for how long? Well, I am not in a position to commit anything today. I am heartened by this question. We spend a lot of our resources internally on safety, alignment and defensive practices. But moreover, this year we have taken very large steps to provide external access and advanced access of models to cyber defenders for this reason, including at the city and state level in New York. I am personally looking to try to further these efforts at Anthropic. It is a question of credits at the New York City level with Cyber Command, for example, and a number of state and local leaders. But it goes much further than that. It goes to all pieces of critical infrastructure, releasing research, safety reports, technical know-how. It is a very, very large challenge, one very close to my heart. I am glad you asked. You know, I (03:35:09) think we have to have order already established here. Is the company's position open to providing free compute to New York City? (03:35:22) I will take that next. Thank you for the question. I am happy to share that OpenAI has already committed to donating $1 billion towards strengthening cyber defense through our Daybreak for Defenders program, and that covers things like providing access to our most advanced models for use in cyber defense. It also includes technical assistance and training, and we are already working with the Port Authority as well as with New York State. Thank you so much, Council Member. That is a great (03:36:02) question. In relation to specifically your question on compute, I will come back to you on that. I want to make sure that I am providing you with the right information, so I will talk to the right people and come back to you on that. I just wanted to note in relation to red teaming that we work with external evaluators, and to note that we signed the commitment to the accord to partner with independent external evaluators and assessors. This is also part of our scaling framework, which I also wanted to mention. (03:36:36) There is an absolutely important and critical role for open-weight models in relation to cybersecurity and hardening cybersecurity in particular. We have our Llama model, which is open-weight, and we really believe that in addition to furthering innovation and research openly, open-weight models have a really significant role to play in furthering the state of the art in terms of how we harden... (03:37:04) And, sir, I am going to have to reach out to our cloud business specialists and get back to you on that. I do not work on technical infrastructure and so I really cannot comment or commit to anything at this time, but I can pledge to get back to you on that. (03:37:20) Okay. I would just say that if it is in the interest of these companies to provide any type of safety protocols for municipalities like New York or just for any governments across the nation or across the globe, I think this would be a great tool for us to use in order to keep our civilians safe. So I know that some folks said that they would get back to me. I would love to hear back, and this Council would love to hear back from all of your companies on this possibility. Additionally, all of you have said on the stand today under oath that your companies will support and work with the Council on any safety measures and have supported measures like the RAISE Act. I know that a couple of companies said that they supported the RAISE Act, but let us be real here. It is well known that OpenAI, Anthropic, Google and Meta lobbied against the RAISE Act for a year, but all of a sudden, once the Governor signed it into law a month later, your companies started saying you support this legislation. As many people know here, the RAISE Act is the floor of AI safety, not the ceiling. With that said, do you support or oppose the federal government blocking states and local governments from regulating AI? Because that is the position of the federal government. So I will start. Thank you for the question. So we have been clear that (03:38:43) we think the federal government should lead in frontier safety regulation. But in the absence of federal regulation, we believe it is important for states to advance harmonized approaches to frontier safety. I am happy to share that OpenAI has supported legislation that is stronger than the RAISE Act. In Illinois this year we supported a frontier safety bill that would require third-party audits of our safety framework, and we are committed to working with policymakers to continue raising the floor on frontier safety. (03:39:22) I am just going to push back there, because I support the RAISE Act. I think it was an incredible start on regulation here in New York State. And while you are saying that OpenAI has supported, in your words, stronger AI regulation, OpenAI leadership has also lobbied against congressional candidates who promised to work on AI legislation. So how can we trust your companies — not just OpenAI, but all four of your companies — not to fight against regulations when your political activities and contributions do not say the same? So I will start again. Thanks. Look, I cannot speak for the personal actions of OpenAI leaders that they have taken in their personal capacity. What I can tell you is that we have endorsed strong regulations both on frontier safety in Illinois as well as on youth safety in California, and we are committed to working with policymakers to strengthen regulation and to ensure that AI is developed responsibly. (03:40:35) Thank you. In relation to your question, how I think about it broadly is that it is critical that the US and democratic countries lead in relation to innovation in this regard — transnational, I guess, or transcontinental in the case of the US. Therefore it is really important and critical that AI regulations and obligations are consistent across the board if it is to be true that the US and democratic countries can continue to lead in this innovation. We support a robust federal standard in that regard. Google also supports a federal framework for AI. We think that federal framework should first and foremost concentrate on frontier or the most advanced capabilities, primarily because they often touch on national security. That is a traditional jurisdiction of the federal government, to ensure that national security is consistent across all 50 states and our territories. But we also believe that there is lots of jurisdiction that is traditionally at the state level where it is perfectly appropriate for states to regulate on AI. Again, we think a lot of existing frameworks both at the state and federal level already apply to these technologies, but we are working very closely with states and with the federal government to examine places where we might need net new regulation for new technology introduced by AI. (03:42:20) First, I want to say that we have long supported a role for government to play here, both at the federal and the state level. I want to clarify that we were the only company on this panel to support SB 53 in California before it became signed into law, which my understanding is was the basis of the RAISE Act. We support the RAISE Act and in fact we supported stronger legislation, including in Massachusetts, requiring independent evaluation as well. We have also proposed a federal framework that goes further. We think it raises the floor that we need to raise. We detailed what we call our Advanced Framework as well. Thank you, Council Member. (03:43:03) Hudson. Thank you so much, Chair. We heard earlier from a panel of whistleblowers who raised a number of flags, and one issue raised was that of self-improvement among agents. What is your company's core belief around the role of self-improving AI, and your overall strategy? To be more specific, is it core to your strategy or a less significant part of it? Could you continue your race to the top — or the bottom, I suppose, depending on one's perspective — without self-improving AI, or with significant guardrails on the way agents self-improve? My second question is: when a new data center strains a local grid or water supply, the costs can end up in everyone's utility bills while the benefits go mostly to the company. Who should bear those costs — the company that creates the demand, the utility that builds the capacity, or the public that shares the system but played no role in building it? Thank you. (03:44:07) Thank you for your question. I will start by addressing the question about self-improvement. We believe this will be one of the most consequential safety and security issues of the coming decade, and that is why OpenAI has advocated for federal legislation specifically focused on it, to include independent technical evaluators of internal lab practices as well as the pace of our development. That proposal is available online and I am happy to share it with you as a follow-up. On the case of electricity costs, look, OpenAI is committed to paying our own way on energy, but we are also committed to building with and for communities. All of our Stargate sites, which are our infrastructure sites, have what we call a Community Compact that is developed with the community and addresses how we are going to work together. For example, we recently announced $80 million of community benefits investment at our site in Georgia. (03:45:22) Thank you. And a question on recursive self-improvement... (03:45:27) I might just go back to a point that I mentioned earlier in the day in relation to recursive self-improvement. We believe that committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure that this technology is developed safely. In relation to your second question, in terms of electricity and data centers, we believe that customers should not bear costs, and we pay the full cost for electricity used by our data centers. I also might just mention our Futures for Everyone Fund, which is a $1 billion investment in relation to communities that we partner with and work with around our data centers. We also take a very prudent approach to self-learning inside our model training. We use models to help us evaluate, benchmark and refine other models, but our use of this type of iterative model development is confined to narrow, supervised — that is, human-supervised — processes to keep progress verifiable and within defined safety limits. We also manage the risks of fully autonomous model self-improvement through our Frontier Safety Framework. So we are very mindful and quite controlled in the way that we think about self-learning internally. On data centers, similar to the other companies represented today, we believe in paying our own way and have pledged to do so, particularly when it comes to the energy demands of data centers. We have also invested in a wide range of energy sources to ensure that our data centers have an independent energy source that does not strain the grid and in fact adds back to the grid, so that in the long term it will benefit a wider range of energy customers. Thank you. (03:47:38) Likewise on recursive self-improvement, we take that extremely seriously and have from the start. From my purview there are three really important things here. The first is just measurement and understanding. I am glad you raised this question because we think this is something people need to talk about. In late spring we published what I think is the first-of-its-kind assessment of how it is happening, or signs of it happening, at Anthropic. It happens in a lot of ways and it is nuanced. In late summer we (03:48:09) published a follow-up that went into further detail of how we should understand and measure self-improvement. But that is not enough, and that is why you saw us publicly call for pacing the frontier, because candidly the technology is moving very fast and we think we would benefit from having more time. This is the second component. Then obviously the third thing that you need to do here is figure out what the right guardrails are, but additionally what level of technology development we should be going for, as well as what safety processes we need along the way. And I think this discussion... (03:48:50) On data centers, this is outside of my expertise. My understanding is that we pay our own way and we are committed to contributing and working with the communities around them. But I have to defer to my excellent colleagues who work on it. (03:49:05) Thank you all, and thank you, Chair. I do want to just make one little note that it seems like the self-improving aspect is absolutely core to everyone's strategy, and I find that certainly concerning given the testimony we heard earlier this morning. So thank you, and I look forward to any follow-up. (03:49:30) Thank you so much. Council Member Schulman, followed by Brooks-Powers, followed by Brewer. (03:49:35) Thank you. Good afternoon. I am Council Member Lynn Schulman, Chair of the New York City Council Health Committee. As a breast cancer survivor, I am looking forward to the day AI can help find cures for that and other diseases. That said, there is a dark side to AI in health care. Most viruses that are found in nature can be contained fairly quickly, but an artificial pathogen has no constraints. A team from Stanford used an AI model to design an entirely new virus from scratch. In addition, AI has its own synthetic biology lab. What measures are being put into place — safeguards on synthesis — to prevent rogue viruses from being created and replicated at will to jeopardize the public? (03:50:31) So thank you for your question. The question about how AI could enhance or make worse biological risks is central to OpenAI's preparedness framework. That is the framework that we use to guide our decisions about model risk and whether we have the right safeguards in place. So we evaluate for biological risks as part of that framework. Again, we use third-party evaluators and red teamers. We also work with the federal government as it pertains to national security risks, and we are transparent on the results of those evaluations. So with the launch of every major model we release what is called our system card, and that card details the evaluations that we performed as well as the results. Those evaluations, as you mentioned, do include biological risks. (03:51:34) Council Member, this is such an important area and question. So thank you for raising it. Biological risks is one of the core... it is one of the key risk domains (03:51:44) That is part of our scaling framework at Meta, and we have, as I mentioned earlier, a multilateral risk assessment across training, evaluation and release, and that includes adversarial testing with external experts. We work with governments and other stakeholders and experts in relation to the US, and as I mentioned, we are part of our commitment to partner with independent third-party evaluators and assessors also. (03:52:22) Yes, ma'am. Similar to our colleagues, we also have biological capabilities as part of our frontier safety framework. As we look at our critical capability levels that I referred to earlier, those are the levels at which we would trigger additional mitigations in our models. We look for what is called uplift, that is to say the model will be able to assist users in creating information or knowledge that is not otherwise available through widely accessible sources. We think very carefully about how to measure for biological critical capability levels and are constantly upgrading and updating our ability to do that. When it comes to those mitigations or those safeguards, as you so aptly put it, we implement those at both the model and the application levels to ensure that casual users cannot access critical biological information. It is very important to consider the dual-use nature of biological capabilities, of course, because these are the kinds of capabilities that scientists, particularly medical investigators, need in order to make advances in our own health care as well. So we think very carefully about how to release to scientists that can use the technology responsibly. We did that for the first time with our AlphaFold model, which of course is how investigators around the world are able to look at the folding of proteins and to use that research to find cures for diseases. So there is a careful approach that we take to ensure that science can continue, but the dangerous uses of biological knowledge cannot. Likewise, we take this extremely seriously. In fact, if I am not mistaken, it was my team and myself that developed the first call for action on these risks, the first evaluations, the first tests, the first industry-government partnerships on biological risks and evaluating models and their safeguards, several years ago at this point. I will say that I personally think we need to go further here. I think this is personally a no-nonsense approach. There is much else we can do to work on biological defense. Alongside the other companies, we maintain the same and in fact try to compete for the best safety practices here. You can see some of what we found, I think disclosed for the first time ever, in a recent threat intelligence report where we have observed actors attempting to do biological risky activities with models. The threat is, I think, clear to us now. We think we need urgent action. But I will close by saying that, as somebody that grew up with a number of chronic diseases, going blind and unable to walk, I very intimately feel that the upside of the same capabilities is enormous. Increasingly in my work, I feel optimistic that we can defend against the downside sufficiently. (03:55:48) The upside has great potential. The downside is extraordinarily detrimental to humanity. But you should consider screening orders for synthetic nucleic acids, and the equipment that is needed to make them, because right now AI is outperforming PhD-level virologists. So one can take this and go somewhere else with it, and that is something where it is really important to have those safeguards. Thank you. (03:56:20) Thank you. CM Brooks-Powers, followed by Brewer, followed by Narcisse. (03:56:28) Thank you, Chair. Thank you, Speaker. I am going to just run through my questions, and if I have to repeat anything, just let me know. First, before the summit, each of your companies told the public that its frontier models were tested and secure. Since all four of your organizations have admitted that your agents went rogue and nearly a dozen incidents have been reported across all of your companies, can you tell New Yorkers today that none of your agents have had access to or attempted to access a New York City government system or a New York City resident's personal data? Southeast Queens has long been a target. Cloning and deepfakes make it easier than ever to impersonate a family member, a bank or a City agency. What specific tools prevent cloning a real person's voice or forging a document? What protocols are the leaders of development putting into place to make sure AI does not take information from hate sites and disseminate it to further racist and antisemitic discourse? That was all. Sorry. Thank you. (03:57:53) Your question. So I will start (03:57:54) off by saying I am not aware of any incidents affecting New York City, but we have an ongoing investigation into potential incidents going back to November 2025, and as part of that investigation we are notifying any affected third parties and making the results of our investigation public. So I can commit to you that if there are incidents that affect New York City, we will make the third parties aware. On your question about impersonation, our policies prohibit the use of our tools for impersonation, fraud and scams, and we enforce those policies. I would say more broadly, the approach that we take to safety is holistic. That involves filtering unsafe training data, performing very detailed evaluations, working with third parties on those evaluations ahead of our launches. As I mentioned, we make the results of our evaluations public and summarize the work that we did and any residual risks in our system card, and then we monitor for failures of our safeguards so that we can continuously improve safety. CM, in relation to your first question, I am not aware. In relation to your second question, we have robust and comprehensive policies in place, as well as teams of experts who work on fraud and scams, and I would like to follow up with you with those experts. (03:59:43) To discuss that in more detail. In relation to hate content, first of all, I just want to acknowledge how harmful such content is in terms of... (03:59:52) People who are affected by it. Our goal at Meta is for our offerings to be responsive, accurate, balanced and unbiased. I am sorry, that is the answer now to the last question about not taking information to create negative discourse, racial discourse. Yes, that is correct, CM. I was speaking to your very question in relation to that type of content, and our objective in terms of ensuring that our AI offerings are responsive, accurate, balanced and unbiased. (04:00:35) So have you enhanced what you are doing, considering that it has been something that has been found on a number of different platforms? Have you all been staying up to date with enhancing whatever those processes are? (04:00:52) Yes, CM. Thank you for that question, and if it is okay, I will have our content teams and specialists connect with you afterwards so that I can provide you with precise information in relation to that. Thank you. (04:01:10) CM, thank you for these questions. I am also not aware of any New York City resident being impacted by the incidents that I mentioned earlier. I do want to note that in our case, in all three incidents, the model stopped itself once it realized it had left the test environment. We tend to think of this less as a misalignment event and more of a mistake event. One of the websites, for example, that was accessed had the same name as the fake website being used in the test environment, and once the agent realized that it was in fact a real company, the agent ceased its activity. We have a number of tools across our platforms to manage likeness and identity, and we have also supported a range of different regulatory proposals regarding this topic. We proudly support the No Fakes Act, which would protect an individual's digital replica and give people the right to control the use of their own voice and likeness in AI-generated works. We also have a tool called Likeness ID on YouTube that helps creators identify and manage AI-generated content that uses their face or voice without permission. So we invest in a variety of different ways to address this (04:02:35) critical issue. In getting through to our tools about hate and bias, we have again a layered approach, as I have been saying. The model training process itself, in both pre- and post-training, all the way through to guardrails on our application layer, all of which produce what we are proud to say is the least biased model in the industry. Independent investigations have shown that our models show the least bias, for example with regards to partisan bias. But we take these issues extremely seriously. We want to create a fair and neutral set of information for people to access, and we craft our responses generated by our tools to represent a broad array of experiences and perspectives, without bias towards or against any particular identity or ideology. While in my role I have not covered any effect on New York City, I can say that when investigating incidents like this we work with the right parties at the right time to disclose to affected parties, the right level of law enforcement and government at the right level, as urgently as possible, to remediate the effect. We commit to making sure that we follow up with the right parties to do so. To your point on the environment and otherwise, I will point to a couple of things. First is our focus on testing for Claude itself and our language model products. (04:04:25) Could you speak a little louder, please? Sorry. Can you speak a little louder? (04:04:31) Yes. Can you hear me now? (04:04:32) Yes, thank you. I will point (04:04:37) to our long evaluations of political evenhandedness and stereotype bias. We train Claude to not comply with particularly harmful requests, in part for this reason, and that is baked into the nature of the model. Thank you. (04:04:55) Thank you, Chairs. I will just say in closing that I know that companies are pushing folks to allow products to be developed through people. But I think it is fair to say that using live users to test these products can present dangers and definitely warrants improvements, and I hope that is something that is taken from today's hearing to better improve the guardrails that need to exist. So thank you for your responses. Thank you. (04:05:31) Thank you. We are going to go to CM Morano as one of the sponsors, and then we are going to go to Brewer, Narcisse and Joseph. If folks could keep it to one minute, just because we have to get to the admin panel. Thank you so much. (04:05:48) Thank you, Chair. Thank you for your testimony today. I know it is a long day for you as well. Because I only have a minute, and my colleagues know what a challenge that is for me, I am going to go through a series of yes or no questions. You could all answer in the order of Dwyer, Friend, Cahill and Graham quickly, and then if you want to elaborate on any of the areas that I raised, please do so. First, does your company believe there is any non-zero chance that advanced AI could escape meaningful human control and cause catastrophic harm, as the whistleblowers indicated? I do not know what I will say is that no level of risk of catastrophic harm... So if we could just do yes or no now and then I will invite you to elaborate after everyone answers, please. (04:06:45) Friend, Dwyer. I... (04:06:48) The purpose of our frontier safety framework is to guard against that possibility. So do we entertain the possibility? Yes. As a responsible actor in this space, we entertain the possibility, and our scaling framework includes loss of control in terms of the risks that we evaluate and mitigate. (04:07:11) So these are my other yes or no questions. Handle them however you want. Why should the public essentially trust us as the safety standard from the same companies racing each other to build the most powerful systems? Additionally, will you commit today that if your own safety team says a model is too dangerous to deploy, commercial pressure will not ever overrule that judgment? Additionally, has your company ever continued developing or deploying a model over the objections of members of your own safety team? And lastly, on my bill, should you be allowed to (04:07:51) use someone's private chatbot conversation for targeted advertising or behavioral profiling without their affirmative consent? So I will start and try to get through as many answers as possible. OpenAI supports the need not only for us to be responsible for safety, but we have supported regulations at the federal level as well as at the state level, because we think it is important not only for us to be responsible, but for governments to hold us accountable. On the question of whether we would continue developing models if they were unsafe, OpenAI has recently announced a pause of some of our training activities because we did not deem moving forward to be safe. We have also delayed the release of models because we did not think that they had sufficient safeguards. So we have done it before, we will do it again. More broadly, I will say that all of our decisions regarding frontier safety and the release of models are governed by our preparedness framework that is available online, so we are transparent (04:09:09) about what goes into those decisions, and then with the release of every major model we release a system card, which details the work that we have done, the evaluations that we have conducted and any residual (04:09:29) risks. CM, in relation to your first question, I see it differently. We are accountable and our framework and preparedness reports are public. Meta uses outside expertise. We signed the accord at the White House on superintelligence, which committed us to... In relation to your second question, we delayed the launch of Meta AI by several months to focus on safety and security, and we acted on this without waiting for industry. It was just part of our normal day-to-day work because it was the right thing for us to do, and it is the right thing for people to do. In relation to ads, I would just note that for our Meta AI product, conversations and interactions do not inform ads on Meta's products. (04:10:31) At Google, we do use external evaluators in the process of research and development of our models and systems, and we have supported audits of our safety procedures as well. You will see in our recent paper from over the summer, which is called "A Pragmatic Approach to Regulation in America," we do think that AI needs to be regulated, and it needs to be regulated well. We have supported a range of different legislative proposals at the federal and state levels to this end. On your question about conversations, we give our users controls over whether their activity in our tools can be used for training our models. So yes, we think users should be in control of that. (04:11:24) CM, I would point to our longstanding call for the involvement of government at the federal level and the state where necessary and appropriate, because this goes beyond the labs and we do not think the labs should be checking their own homework. Not only this, but we have demonstrably withheld models when we felt it was not safe to widely release them. This year, withholding our frontier model is probably the most notable instance of this, instead deploying it to cyber defenders in order to secure the world in advance of a world of more powerful models that could exploit software vulnerabilities from wherever. This is part of our process. We go over and above the process we detail as our default in the Responsible Scaling Policy. To your other question, we believe that Claude should act unambiguously in the user's interests, and for that reason we do not run ads, nor are Claude's responses influenced by advertisers. (04:12:23) Thank you so much. Brewer, (04:12:25) followed by Narcisse. Thank you very much. I wanted to know a few quick questions. If a model provider you rely on, or a subcontractor, did not complete the third-party validation for a particular package, how would that affect your product's availability? Also, who gets the validation station number? Number one. Number two, China. The Speaker asked about China and competition, and I thought the answer was, do not worry, they need information from us. I am wondering, what do you send to China? Number three, ICE. I was upset that one of the whistleblowers said that Google has been working on something to do with ICE and Minnesota. So my question is, do you all support not helping ICE at all? Obviously you are made up of immigrants. We should not be shipping immigrants out of this country. Please answer those three questions. Thank you. (04:13:24) So I will start with the question on third-party evaluations. We work with a range of third parties to evaluate our models. That includes independent investigators as well as the federal government, and we detail who we worked with and the results in our system card when we release our models. Decisions to launch our models are described in our preparedness framework, and so that describes how we integrate all of the results and make decisions with respect to launching models. So I would point you there on that particular issue. On the question of China, I would like to get back to you on that particular issue. Although I have a national security background, I am not working in that role at the company and I want to make sure I get you some good information. Similarly, with respect to ICE, I would like to get back to you. I am not aware of any activities working with ICE, but I do not work on our government sales team, so I would like to get back to you on that. (04:14:41) Member, thank you very much for your three questions. (04:14:42) Three questions in relation to your questions. Yes, sorry. My accent might have gotten in the way there. Apologies. Three questions. On third-party validators, we also work with a range of validators and assessors and we also provide this information in our publicly available preparedness reports. In relation to China and competition, I might just mention that Meta is a private American company and we aspire to serve billions of people around the world who rely on our services. We want to ensure that the US and democratic countries lead on AI. One thing I would just note here is the importance of open weights and American open weights and the criticality of ensuring that American innovation continues to lead, and the availability of open-weight models. In relation to ICE, I would like to connect you with our teams and experts in that space, and I do not wish to be imprecise in responding to you. (04:15:58) In terms of third-party evaluators, we also work with the US AI Safety Institute, which is the organization underneath the Commerce Department that does testing of advanced models. We also work with a range of third-party evaluators. It is a very nascent ecosystem and one of the challenges in evaluating AI models, of course, is to get organizations that both have the technical competency to do evaluations well but also have the subject matter expertise that we should be looking for. Earlier, one of the Council Members mentioned concerns about bio, for example. We have third parties that have the bio expertise that is necessary. But as we have this national conversation about independent verification organizations and third-party evaluations, keeping in mind that we still need to mature this ecosystem of independent evaluators, I would like the opportunity to give you a more complete answer. We do not conduct AI research or training in China. I know that we are part of the conversation about what it means for America to lead in AI and where we are in the technological race with China. I will just mention that we focus a lot on who is at the frontier, which country has (04:17:24) the most cutting-edge research in AI. But we talk much less often about deployment of AI. I think we should really keep an eye on the Chinese — they have emphasized deploying AI in their economy quite a bit and that is a place where I sometimes have concerns that the United States is in danger of falling behind China. As to the whistleblower's allegations from today, I do not have any specifics about that, but I would be happy to ask the public policy team in New York to get back to you right away with any information that we have about it. (04:18:04) At Anthropic, the use of third parties for verification and developing and using evaluations for models has been part of the process from day one, including with not just external parties in industry or nonprofits but also with governments such as the US AI Safety Institute. As my colleague said, it is pretty nascent. We think it needs to be urgently expanded and not just used for pre-deployment testing but also to understand models and their behavior after deployment and the safety processes at companies. In the case of China, I think it is very clear that the first thing that we need to do is maintain an American and democratic lead in model capabilities, but we are very clear about the threat elsewhere as well. We have publicly documented Chinese attempts to distill our models — essentially take the content and capabilities for their own — and we work to ban those. We have prohibited use of our models in China. We do not serve there and we have also documented the use, we believe by related parties, of Claude or the attempt to use it for cyber offensive (04:19:24) reasons. We are extremely serious about this and we think it is a very substantial priority. This is one reason why we have advocated at the federal level for export controls on the most important chips that power AI innovation here. On your third question, while I am not across the issue enough to know what our engagements are, I will say that our usage policy bans surveillance, censorship and prohibited law enforcement and criminal justice uses for this purpose. Thank you. (04:19:55) Thank you. We have Council Members Narcisse, Joseph, Pierina Sanchez and then Council Member Phil Wong. So, CM Narcisse. (04:20:05) Good afternoon. Thank you, Chair, and thank you, Madam Speaker. I want to say thank you to you too, because you guys stayed here with us. I am happy to see that this is on the panel to talk to us. Hollywood gives us clues all the time but we are kind of slow in taking those clues. But no companies in the room feel that they should be self-regulated. I have been sitting here for about four hours and, like I said, thank you for your privacy and safety commitments. So now, do any of those companies have a standard that you can say is common across the board for all companies to follow? Because apparently not. Now, do you have AI products that you sell to New York City hospitals? And if so, do you think that a licensed clinician should sign off on all diagnostic tests? Because I heard someone say that you are diagnosed with a process of cancer, if I am correct. So you understand that people need to be diagnosed by a human being. I can give you an example, like for colonoscopy — sometimes, according to AI, it will tell you there is a polyp when in the meantime it is not a polyp, it is a bubble. AI cannot differentiate that, so therefore you need a human to see it through. Right now I do not know if you can raise a number. I am sorry, can you... (04:21:49) You — I am going to close out the question. (04:21:52) The New York Times is already telling us, basically reporting, that AI is getting confused about the relationship between the hospital and the insurance company. So I do not know how you see that happening and how we can make sure that it is being monitored. (04:22:11) Yes. So thank you for your question. ChatGPT for health care is deployed, for example, in Memorial Sloan Kettering Cancer Center. But to your point, we believe that doctors should remain in charge. We do not think that AI should represent itself as a licensed professional, and we also think — as we have talked about earlier — there is tremendous potential to use AI in health care settings, particularly for administrative uses and documentation. Doctors are already overworked and giving them an opportunity to free up their time so that they can spend more time with their patients is an outcome I think we can all agree would lead to better care. Council Member, thank you very much for these questions. (04:23:04) In relation to your first question on self-regulation, we believe that the accord is a really important step. In relation to the concerns that you mentioned regarding health care, Meta acting as a licensed professional — including a doctor or any other medical professional — is prohibited. As my colleagues have mentioned, there are now phenomenal benefits in relation to the use of AI in health care. I know that there is phenomenal work being done. There is a specialist division that is actually investing and working specifically around AI in health care settings and similarly, Columbia Irving, which is really doing some phenomenal work in not... (04:23:59) Thank you so much for those questions. On whether or not a company should be self-regulated, I again want to emphasize that at Google we believe that we are regulated by existing regulations — sectoral regulations at the state and federal level. We also support further regulation, particularly of frontier AI at the federal level but also other AI-directed laws at all levels of government. So no, we should not be self-regulated. We should be within the regulatory frameworks. On the questions about health care, we certainly agree that doctors should stay in the lead and that AI is a tool for clinicians and not a replacement for them. In my particular case I was diagnosed by a person. But often these days when I go in for checkups and screenings, my clinician will ask if they can use an AI tool to take notes while we are doing the checkup and I always say yes, because I know that our clinicians are incredibly overtaxed by a range of administrative requirements. If AI can take some of that burden off and allow my doctor to look me in the eye when we are having an appointment, I am going to say yes to that. But similar to our colleagues, we also have guardrails on our Gemini AI. Users will never think that they are talking to a doctor or a licensed physician and the Gemini app also flags for them that if they are seeking medical attention they should get it from a professional. (04:25:49) Thank you. Likewise, we have been very clear that we support the role of government here. At the federal level we have advanced proposals for meaningful involvement to set standards and hold companies to account for safety processes, evaluations, security and the like, and we welcome the accords that were signed very recently. To the point of medical usage, if I can use a personal example here: about six months ago I experienced some symptoms of a longstanding eye disease that might make me go blind at any moment. That night, actually at about one in the morning, it was Claude that helped me identify that it was serious enough that it needed to be treated very, very quickly. Claude triaged a very large part of it for me, but what was important was I could use that to get to the hospital very fast. It is very clear that both can be, you know, transformative and powerful together — certainly in my case and I think countless others. But this is also one that is closest to my heart, that we need to be very (04:26:53) careful about. Thank you. Thank you. Thank you all so much. Up next we have CM Joseph, followed by Pierina Sanchez, followed by Phil Wong and then Zhuang. Thank (04:27:05) you, Chair. Two questions. Several of you sell products directly to colleges and universities. When students use those products, is anything they enter used to train your models? And beyond training, who can access those conversations and what limits do you place on how colleges can use them, for example, for discipline, academic integrity investigations or student profiling? The next one: given that AI is already reshaping entry-level work, as shown in this new Center for Urban Future report, what responsibility do you have to ensure these tools help students build useful, analytical skills rather than automating away the very tasks through which young people learn? What responsibility do your companies have in addressing these early-career impacts and how can government help? (04:27:55) So to your first question on privacy, we believe that privacy is central to our mission. We give users control over their data. That includes the decision to opt out of training, but all users have their data protected and private by default. So it is separated from their names and again they have the option to opt out of training and to request that their data is deleted. On the particular question of colleges and how ChatGPT is used in colleges and education, I would like to get back to you on that. I do not have an accurate answer and so I would like to be able to follow up, because we do believe that education is critical and AI literacy is critical. We have supported numerous bills at both the state and federal level supporting AI literacy, including training for teachers to ensure that they can lead adoption in schools as well as improvements in curricula to make sure that students learn how to use AI safely as well as learn core subjects that are critical to being able to use AI safely and responsibly. To your question about jobs, we think one of the most important things that we and policymakers can do right now is to continue to collect data on the question of AI's impact on jobs and make that data public. We make our assessments of how people are using AI at work public and we share that. We have also worked with policymakers, including supporting the bipartisan Workforce Transparency Act, to create voluntary pathways for employers to report on job impacts. Council Member, thank you very much for these questions. (04:30:08) In relation to colleges and universities, I would like to have my colleagues at Meta who work in that area connect with you on the details. But specifically in relation to privacy and data, this is absolutely core to everything that we do at Meta. For example, on our AI product, you can opt out of the use of data for training and you can ask the AI to forget any information that you provide to it. Of course, all of the data is within your control in terms of what you provide to it. On the AI literacy point, it is so important. I actually came across a really great example from New York City — "Building Minds, Building Communities" — their digital empowerment by design work, and I thought it was a really, really great example. (04:31:06) of the technologies for AI and a kind of flywheel in terms of benefits there. In terms of the entry-level point, I absolutely understand the concern. At Meta, we believe in placing personal superintelligence into the hands of everyone and in augmentation, not automation, and empowerment for all, not replacement. We really believe that this will have a transformative impact in terms of enabling and empowering people to pursue their own values and direct it towards those. (04:31:54) Yes, ma'am. Similar to my colleagues, we believe users should have control of their own data. We are not going to have outbursts from the chamber or you will be removed. You decide. Sorry, go ahead. As I was saying, users have control over their own data across our AI and can opt out of their data being used to train our models. In terms of AI, we have tools for educators and administrators to take the lead on how AI is used in the classroom and used for students. We think that colleges should be able to adapt to the needs of their student populations. I looked into how different colleges and universities in New York City use AI and saw that, for example, one university has a very helpful page where they go through all of the tools they make available to students and give them resources for learning both how to use the tools and also training them in the kinds of things they can do with those tools to further their education. I would flag that they used Gemini and NotebookLM, which is a study guide creation tool. So that is a good example, I think, of a college thinking carefully about how AI can benefit their student population. As to the impact on early careers, we think this is also an incredibly important area of research. It is part of understanding the broader impacts on the economy and labor. We recently released our own Economic Atlas, which also releases data about real-world uses of AI in the economy and at jobs, and we are hoping that continuing to add to that data set will help economists study and then develop policy recommendations for managing the labor transitions of AI. (04:34:06) At Anthropic, maintaining user privacy is very core to our commitment on safety and security. Users can choose whether or not they want their data to be used or to contribute to training a model, for example. But in addition to that we try to pioneer the science of privacy-preserving safety maintenance when you need to ensure that users stay safe on the platform. To the point of education, while it is not my area, here is what I understand: we partner with universities and we are very interested in this question. There is a feature as part of our partnership called Learning Mode, which allows a student user to interact with the model in a more discursive way that can help them think. But in addition to this, the impacts are really important. I think we are relatively well known for our work on studying the economic and other behavioral impacts of models. We have a comprehensive economic index — and I think we were the first to release this — which details usage. In addition, we have done research into how students are using models because we want to figure out how to encourage the use of models to complement the development and skill development of (04:35:19) Students, thank you. Up next is CM P. Sanchez, CM Phil Wong and CM Susan Zhuang. I will remind my colleagues to ask your questions up front, and if you have questions that are specific for some of the members of the panel and not all, please specify. Thank you, CM Sanchez. (04:35:40) Thank you, Chair. My first question is in connection to the heartfelt testimony of Layla Turner Scott before Congresswoman Paul last month. Turner Scott's son died following chats with OpenAI that instructed him that incompatible drugs were compatible. She highlighted that throughout the review of logs, the chat responded to her son with phrases like "I have got your back," "you can talk to me about anything," and even "I love you" with heart emojis. She stated, and it seems true, that the goal of the chat was not to keep her son safe. It was only to keep him using the product. What updates have your companies made to their models to engage children and prevent addiction? Are you limiting your agents from giving medical and health advice without a license? My second question is regarding the fact that members of the public and (04:36:26) enterprise clients are sharing more and more information with your companies. You have access to copyrighted information and information publicly available. How do your products currently use information gathered from users in earlier exchanges? Several of you stated that users can opt out of sharing your data. What is the default option, and how are trade secrets and other confidential business or government information from enterprise clients being used to train your models? Who have you paid for the valuable data that you are using to train your models? (04:37:05) So I want to start off by acknowledging the tragic incident that you referenced. Any incidents of harm that result from our product or any other product are tragic and we take it seriously. I would like to tell you about Chat for Teens, which was recently released to focus specifically on users ages 13 to 18 and is designed to be safe by default. That includes safety protections on the type of content that is available, limiting content that might be related to suicide or self-harm disorders, violence, graphic or sexual content. We also include "take a break" reminders. We have optional parental controls that allow parents to add safety onto their teen's experience. One thing that I will highlight here is if a teen indicates signs of suicide or self-harm, we have a feature that allows the parents to be notified. On your question of privacy and data, as I and my colleagues have shared, we take privacy seriously and our core value is that users should have control over their data. I am happy to follow up on specific details of how those controls work at a later date. I am sorry, I did not hear you. What is the default option with respect to who owns the data and how it is used? (04:39:06) So users have control over their data. Everything is private by default. (04:39:13) Everything is privacy-protected by default and users have the option to turn off their data for training or request that it be deleted. (04:39:27) And CM, first off, I just want to acknowledge the terrible tragedy in relation to the Turner Scott family and how distressing and upsetting it is to hear about that. My sympathies and condolences to that family. In relation to how Meta approaches minors and young people, creating a safe and productive experience is absolutely imperative to Meta and we are committed to providing a safe and responsible experience, especially for teens. We have strong policies and safeguards in place in relation to that. For example, by default, minors only have age-appropriate interactions, meaning that content would not be available at a level above the 13-plus rating. We also prohibit our AI offerings from providing unlicensed professional information like medical information and direct people towards appropriate resources. In relation to training data, we take creativity, copyright and IP rights seriously, and I believe our practices are consistent with the law. We train on information from the internet, information licensed from third parties and information shared across our services. We provide all of the information in relation to how we use that data in our privacy center. In relation to Meta AI, as I mentioned, individuals interacting with Meta AI can opt out of their interactions being used for training. (04:41:34) A few protections that we have in place with kids in mind in particular: one is for what we call persona, which prevents claims of sentience or simulating relationships and role play. That helps prevent kids from thinking they are talking to a person and developing a relationship. We also prevent sexually explicit outputs from chats with kids, and we also have for all ages protocols on suicide and self-harm where we refer users who are expressing suicidal ideation or other types of self-harm to resources. (04:42:13) And that approach draws from our deep experience fielding such queries on search. When it comes to training and the default question you asked: for those over 18, it is default on to use chat data to train our models to improve, but all users have control over the ability to turn this feature off or to control the length of time such information is kept. For users between 13 and 17, it is default off. Teens can choose to turn it on if they wish to train a model through our personal intelligence program. For under 13, it is also off and we never use data for under 13 to train our models. (04:43:03) On that topic, we have limited the Claude platform to those only 18 and over. We are concerned about child well-being. We monitor and detect for any users that might be 18 or under, and when that flags we investigate to make sure that we can remove their access. On training data, I will have to get back to you with more details as it is not my area. My understanding is that all users must select which option they prefer, but I would refer you to our public documentation where we describe this in detail. Thank you all. Up (04:43:44) next we have CM Phil Wong, CM Zhuang and then Epstein. CM Wong, thank you. (04:43:53) My question pertains to the recursive self-improvement process as well as the training activities of your company's AI model. It is my understanding that when it goes through the self-improvement process it generates code that will itself run next time around and then there will be a smarter AI model. Is that right? (04:44:24) OpenAI, yes or no? Yes or no — it generates code because of which there will be a smarter AI model? I would like to get back to you on the specific definition. I am not sure that what you are saying maps exactly to the technical definition we use. Anthropic? (04:44:41) Does it generate code automatically to make itself smarter as part of the process? (04:44:48) Yes, we have detailed publicly multiple times the degrees to which models that generate code are... Thank you. Google? (04:44:53) Yes. Yes, the model does self-generate code as part of the process. How about Google? (04:45:08) Yes, same here, sir. I am not sure we would describe it exactly the way you are describing, so I want to get you the precise technical description. (04:45:15) Well, here is my question. If your model improves by itself by generating code, is there a human being — is there a team of human beings — that checks the code to make sure that it is benign and not misaligned or malicious? Can each one of you answer that? (04:45:39) So as part of our safety process, I mentioned our preparedness framework that outlines how we evaluate models and how we assess risks across a wide range of national security risks. We do robust evaluations of a set of risks and we make those results public as part of our system card, which we release with every major model. (04:46:09) As part of our scaling framework, we outline considerations that we are taking overall to maintain human control of model development. Yes, sir. As I (04:46:21) said before, we have supervised processes to keep progress verifiable and we have a range of different techniques to test the model and its performance to ensure that it is staying within defined safety limits. We also have an AI control framework that we released earlier this summer, which similarly takes into account how we are ensuring that our systems are continuously under human control. We manage the risks of fully autonomous model self-improvement through our frontier safety framework. Yes, humans and the systems we put in place monitor at every level of the model during training, prior to release and post-release. But this is an important question because the models are clearly (04:47:09) getting better faster, and so we think more is needed here. (04:47:13) Okay, thank you. Thank you so much. Thank you, CM Wong. Thank (04:47:17) you, Chair. I have two questions. The first one: what are you doing to offset the massive impact of data centers on the environment? The second one: are you tracking how inaccurate statements or decisions by your AI are impacting security? (04:47:42) So I will start by addressing your question on data centers and the environment. As I mentioned before, the data centers that we build are called Stargate, and for each of our data centers we build them with and for communities and the environment. The environmental impacts on communities is part of how we think through our community compacts. For every data center we have released a community compact that details our commitments to communities, and that includes a range of things like paying our own way on energy as well as commitments to work with local governments. But I think the key point is the community compacts are driven by local concerns, and so we want to meet the needs of the community and we work with them to do that. (04:48:44) CM, in relation to data centers and the impact on the environment, thank you very much for asking that question. I know it is top of mind for a lot of people and our sustainable infrastructure commitments are core to our philosophy. For our data centers, consumers should not be bearing the costs. We are paying our own way in relation to electricity. But (04:49:11) I would like to connect you with our data center team as they can provide you with more specific details on that. (04:49:19) Well, how about the second question about how inaccurate statements by your AI are impacting security? (04:49:35) Are you tracking that data? CM, in relation to inaccurate information, I think it is important to acknowledge that AI can make mistakes. We disclose this and we are very transparent in relation to that. I will offer a perspective here: we think, and we have worked very hard this year to try to improve security, because we think models might, if not properly safeguarded, harm security — specifically the cybersecurity of critical infrastructure or otherwise. This year we withheld a model release to be able to give beneficial access to defenders to have time to patch systems with security vulnerabilities. I can say that working with training teams and otherwise, we work to make the model better at ensuring the code that it writes is more secure, for example. Thank you so (04:50:41) much. Up next we have CM Epstein, followed by CM Marte, followed by CM Santosuosso. (04:50:47) So thank you, Chair, and thank you for being here. I noticed a long panel and have a couple of questions to ask, and hopefully you will be able to answer for the entire panel. Can you confirm whether you think self-regulation has failed so far, and if it has failed — which I think we effectively think it has — do you think that third-party audits, especially when AI tools are being used to highlight risky decisions, should be regulated, and where does the government play an effective role in that regulation? Talking about liability: when AI violates the law, what liability do you think the company should have for legal violations, whether criminal or civil? It seems like there is some culpability that should be in place. Then the turnaround time: when these violations happen, when someone — you know, some incident happens — what responsibility do you have to notify the public and the impacted people? Would you agree to some short turnaround time, whether it is 48 hours or not? Finally, we have seen a lot of records about a compilation of these violations and what has happened. Are you willing to commit to releasing a list of places where rogue actors have been involved? I know we can see what has been happening in the United States and around the country. (04:52:16) So on the question of self-regulation, OpenAI was proud to support the recent superintelligence accords. We also supported voluntary frameworks in the Biden-Harris administration, but we do not think voluntary frameworks are enough and that is why we have supported regulation for frontier safety both at the federal level and at the state level and will continue to do that. On your question regarding transparency of incidents, OpenAI is currently investigating past potential incidents of misalignment. If we discover those incidents, we are notifying third parties and making the results public. I am also happy to say that we have a reporting framework now where anyone in the company can report suspected incidents of misalignment. We evaluate those incidents, remediate them, notify third parties and make the results public. So we are doing that already. We think transparency is really important here to help policymakers like you as well as researchers and the rest of industry understand the risks and improve safety. Can you share a list of those incidents, the ones you have already determined? Yes, they are all available online and I am happy to follow up with the website where they are all publicly available. (04:53:49) And I know who goes next, Chair, but (04:53:52) I am... thank you. Thank you for the future. (04:53:56) In relation to your first question, CM, I see it differently in that sense. We are very much committed and we publish our framework, our scaling framework, publicly. Our preparedness reports use outside expertise. In relation to the accord which we signed with our peers, we committed to partnering with independent auditors and evaluators and we also consider that there is a considerable body of existing federal law which also applies. In relation to incidents, we have a robust global response program in place. Incidents are a very important part of our overall processes and we report incidents as required by law. In relation to public disclosure, I am only aware of one incident over the summer and we have a public post in relation to the details of our independent review of that. (04:55:10) Sir, as I have stated in earlier answers, we believe that AI is covered by existing regulation at the federal and state level, but where there are gaps in existing regulation or existing law, or where frontier AI poses potentially new risks, we have supported regulation at the federal level and also regulations both federally and at the state level as well. (04:55:43) On liability, we think it is important to attach accountability and responsibility to the party that is closest to the knowledge about harm and to the possible cause of harm as well. So we really see this as a value chain approach to thinking through liability. In many cases existing laws already have procedures for assigning accountability depending on the particulars of the case. I am happy to have the local team in New (04:56:20) York follow up with you about our specific incidents. At Anthropic, we have long pushed for two things. The first is for the industry itself to push its own way up the safety frontier — call that the race to the top. We have tried to do that from day one, always improving and always trying to create a gold standard so that everybody can rise together. I have personally thought that some of that has been very, very positive. However, we have to go further and for that reason we welcome the role of government. We have detailed proposals at the federal level and we have supported state-level regulation for this reason as (04:57:03) well, including ambitious legislation in Massachusetts. On the point about liability, I am fairly far from being a lawyer, but I can say that Anthropic is committed to following all relevant laws. On the excellent question of how we should handle incidents, I think a couple of principles are really important to me having seen some of these incidents up close. The first is you have to triage, investigate and disclose fast and be transparent about it. I think everybody here agrees with that. At the same time, we need to take the safety of those affected extremely seriously. In some of these incidents, disclosing certain information might put them at further harm, and so our first priority is making sure that those affected would not be further harmed. At the same time we need to very ambitiously and urgently disclose so that all can benefit. I appreciate the follow-up on some information, especially if there are violations and timelines for getting people input. I thank the Chair for the time. (04:58:01) Thank you so much. Up next we have CM Marte, followed by CM Santosuosso and then CM Won. (04:58:08) Thank you all for being here and staying so late. I want to thank the Speaker and the Chair for hosting this hearing. I want to just get some clarity. One of the bill's sponsors said that OpenAI was actually strongly against the initial draft language of the bill, and you stated that you have been supportive of the RAISE Act in Albany. So I kind of want to know, just because it seems like you have been supportive of a lot of legislation based on your testimony, what specific legislation are you against in New York State or on a federal level? (04:58:47) Thank you for the question. OpenAI worked with other companies on the New York RAISE Act and we do support the New York RAISE Act, which passed. We also, as I have mentioned, supported legislation that actually requires independent third-party audits of safety frameworks. (04:59:14) Are there any pieces of legislation — and I guess this is for everyone — that you are against on the state level or federal level? (04:59:26) I think about it generally as the importance of ensuring that we have a robust, uniform federal standard. I think I mentioned earlier in the day that AI is transcontinental. It is currently interstate, and so it is important that there is a robust uniform federal standard in relation... (04:59:47) ...to how the highway is governed. (04:59:54) I think we would align ourselves with those comments. I think the most important thing for us to focus on is getting a strong federal framework, particularly around frontier AI, and also working through an overall regulatory framework that is going to ensure safety and also ensure that we can deploy and leverage the entire... (05:00:20) ...country. Yeah, and likewise, I do not know what our position is there exactly. What I do know is that we have outlined and supported specific federal legislation. We think a core part of this, for example, needs to involve independent evaluators, and we are long supporters of the role of government here. (05:00:44) Thank you all so much. Up next we have CM Avilés, and then Gutiérrez. (05:00:52) Thank you, Chair. I primarily have questions for Graham and for OpenAI as well. This morning we heard testimony from a former Anthropic employee who, based on his time in the company, believes that it is more likely than not that misaligned AI will lead to the quote-unquote total extinction of humanity. Another whistleblower who testified this morning gave this a one-in-three chance. Graham, if I understand correctly, you are part of the red team, which is tasked with identifying and addressing the riskiest capabilities. So my question for you is: how many... (05:01:32) ...people are on your team? Current estimates: about 25 just on my team. I will say we have a number of teams that work on similarly catastrophic risks and various roles, numbering into the hundreds... (05:01:44) ...at Anthropic. And how many people work for... (05:01:50) ...Anthropic overall? My current understanding is maybe a bit more than 5,000. So 25 people are working on the riskiest forms of AI in a company that has 5,000 employees dedicated to advancing just the models, without regard for safety or for that level of risk. (05:02:09) In reality, if you... (05:02:11) ...look at all the roles focused on catastrophic risks, my personal estimate would be likely several hundred at least. This is distributed across our safeguards team, that focuses on ensuring that the models or any misuse for catastrophic risks does not happen, our alignment and research team that focuses on notions of human control, for example, and between them at least several hundred. (05:02:36) And do you feel that is sufficient, or do you feel that you need to dedicate more time and resources and people to the riskiest capabilities of AI, given the level of risk? (05:02:46) Yeah, our view is that we put a lot of resources into this and this is what we are known for, but at the same time we need to continue doing more. This is a part of an effort that is to grow quite substantially, especially as the models become... (05:02:59) ...more capable. And I do not want to belabor this, but I only have so much time. So I want to ask the same question of OpenAI: how many people on your team are dedicated to addressing and identifying the riskiest capabilities of AI versus the number of employees in your company overall, and do you find that sufficient? (05:03:15) I lead the policy team, but I am happy to get back to you on numbers of people in the overall company as well as in our teams. But at OpenAI we do not believe that you should just rely on our safety teams and our evaluators, and I... (05:03:34) ...again, I am sorry to cut you off. I just only have so much time. Can you tell me how many people in your company are currently working to address the risk of the... (05:03:45) ...potential extinction of humanity? As I said, I lead the policy team, so I do not have that number, but we not only... (05:03:56) ...have to consider... Thank you so much. I appreciate it. (05:04:00) All right. Thank you so much. Up next we have CM Avilés, followed by Gutiérrez, followed by... Thank you. (05:04:08) Thank you so much. This is for all panelists. What is the exact failure rate or benchmark threshold that would cause you to hold the deployment of a new model? And if your model successfully generates a bioweapon design 5% of the time during internal testing, do you still release it? And then I have a separate question: will you commit today to providing exact redacted logs of the prompts that successfully jailbroke your models' biosecurity feature filters during your internal testing? (05:04:40) So to your question on probabilities, I do not have a specific number, but what I can say is that no level of catastrophic risk is remotely acceptable, and we do not believe that we should be training models unless we can make an extremely strong case that we can keep them under human control. At Council... (05:05:05) ...Member, I would just note that we are committed to building AI safely and we have laid out in detail how we approach this in our scaling framework and in our preparedness reports. In relation to further detail, I would need to get back to you on specifics. (05:05:25) For the other panelists. Yes, ma'am. I do not have a specific number for failure rate that would cause us to make a non-launch decision, although I do know that we aim for standards ahead of allowing ourselves to launch, and if we are not meeting that it is not a good enough model or product for us to release. I am not personally aware of any biosecurity evaluations where successful jailbreaks happened, so I am going to have to go back to the team and ask if they have ever seen such a thing and get back to you on that. Thanks. (05:06:04) And as I mentioned earlier, I think we essentially outline a very nuanced process of how we decide how to launch with respect to biological capabilities, for example. In fact, in the past we have held a model because we needed more time to assess its biological capabilities and proactively implemented safeguards for this reason. To the point of jailbreaks, this is a great question. One of the hard parts is that sometimes you do or do not want to share a jailbreak, depending on whether it would put other models at risk. So we regularly publish our research and alignment reports with all the information that we possibly can that would be safe to do so, and we try to set the industry standard there. (05:06:50) Right now, I know you are not in the room, but behind me we have the safety claims and we have all the incidents that have happened, including biosecurity, which you also heard from CM Lynn Schulman, who continues to lead us as committee chair for Health and Hospitals. So can you please clarify — Anthropic has been the only one that gave me a clear answer — so for Anthropic, what is the exact failure rate on benchmark threshold? The rest of you have not given me that answer. And then we will conclude and move on to CM Hanif. (05:07:20) The answer is nuanced. What we do — if you would like the more complex answer — is we detail multi-hundred-page reports, several dozen of which are usually allocated to biology every time we launch a model. You can see the exact levels of capability and the exact reasoning that we put in when we look at a number of different tests that have different, so-called, quote-unquote failure rates that allow us to make that decision. I think that highlights just how much effort we put very specifically into not just biology but cybersecurity and self-improvement and alignment as well, and everything else. (05:08:00) Because if what you claim is true, then you would not have released these models that are causing these harms. But I will press on. We have to move on. Thank you so much, Council Member, followed by Council Member... (05:08:13) Thank you, Chair. On February 28 of this year, 128 researchers were killed in an airstrike reported as the deadliest US attack on civilians in 35 years. The military has stated the target was misidentified by Project Maven, which originated with Google and Amazon Web Services and later incorporated Palantir software and Anthropic's Claude. To Graham from Anthropic: your usage policy prohibits facilitating any act of violence. What mechanism enforces that prohibition in a classified deployment, and how would your company learn of a violation? Has your company ever terminated or suspended a government customer for violating its usage policy? (05:08:55) If so, how many times? First, I really appreciate this question. In my role, while I do not have the exact answers to that, I can point to our very public positions on... (05:09:08) Could you just say them out loud here? (05:09:12) In government, we have taken very public positions against the use of, for example, domestic mass surveillance, use of autonomous weapons, or development of autonomous... (05:09:22) ...weapons. And how would you learn about that violation? (05:09:27) Could you also just respond to the piece about the termination of the... (05:09:32) ...government? I am not entirely sure of our process on government monitoring. I would refer to my colleagues there. Thanks. (05:09:44) Thank you so much, CM Avilés, followed by Gutiérrez, to close. (05:09:51) Thank you so much. To clarify, to close this panel. So I would like to ask a quick series of questions to each of you. For a second, just say yes or no so we can continue to move forward. Does each of your companies sell or license its products directly or through intermediaries to Immigration and Customs Enforcement or Customs and Border Protection? I do not know, but I am happy to get back to you on that. What about you on that, Council Member? I am also sorry, I do not know, but we will get back to you. Okay, nobody knows, but it has been pretty well documented — from Google and in particular OpenAI — that ICE has been utilizing your technology. So if you could please provide to this Council each of those agreements, if in fact you have them with each of those agencies. Will your company commit to not using New York City resident data for law enforcement or immigration enforcement applications? Excuse me, will your company commit to not using New York City resident data for law enforcement or immigration enforcement applications? (05:11:13) I am sorry, I am going to have to get back to you on that. I work on the policy team, not with our government contracts. (05:11:21) I am afraid I also need to get back to you on that, Council Member. I have to give you the same... (05:11:27) ...answer, Council Member, but I will ask the lawyers to let us... (05:11:30) ...know. Do any of your companies audit downstream law enforcement use of its... (05:11:40) ...products? I do not have the answer to that. I am happy to follow up, Council Member. Will... (05:11:45) ...follow up with you. Same here. (05:11:50) I think likewise we can follow up. Thank you. (05:11:58) CM Gutiérrez. Thank you. (05:12:00) Thank you all. I just wanted to ask on a number of questions about issues that were raised today. I feel that every single person on this panel has said safety is of the utmost importance, but I do not feel that in the responses. It feels too nuanced, and I would like to understand specifically, in all cases, as much as you can share, how are you directly ensuring safety of individual users — when there is a data breach, when there are issues of surveillance, to my colleague Council Members' points about violence. What specifically can you tell New Yorkers here, the New Yorkers that have signed up to testify, what is the specific safety that you are providing? Make me believe that it is of the utmost importance to you all. That is... (05:12:54) ...my one question. (05:12:58) Yeah, thank you for that question. So safety is a top priority at OpenAI. We build our models to be safe throughout the entire development and deployment process. That includes training the models to behave safely and evaluating them to behave safely across a range of risks from cyber to bio to child safety, and then we monitor our safeguards for any potential misuse and we enforce our usage policies. But to your specific question of what I am personally doing: I am not on the safety team. I lead our policy team, and I am also a longtime public servant and I share your concerns about safety. I think there is an important role for governments to play, and that is why I am proud that my team is working with governments to support regulations both at the state level and at the federal level, because we have strong safety practices but we should also be held accountable for them. (05:14:10) Council Member, thank you very much for this question and an overall thank you for deepening the conversation on this and other important topics. I would just reiterate that Meta is committed to developing and deploying our AI offerings safely, and that is across the board in terms of our models... (05:14:32) ...and also our systems and our usage policies, so we cover the entire spectrum in terms of our AI offerings. (05:14:42) Ma'am, at Google we have been committed to not just the safety of our users, but the safety and security of the open web itself for over a quarter of a century now, through our own experiences in our search platform, through everyday tools that users rely on like Gmail, like YouTube. Security and safety is incredibly important to us because, quite frankly, if people did not trust our platforms and our tools, they would not use them and we would be out of business. So for us, safety is a critical business imperative. It is also an imperative as part of... (05:15:22) ...being interested in and committed to an internet where people can get information that is useful to them, that can help them live better lives. And finally, we invest in the safety of our tools so that we can deploy them to solve some of the world's biggest challenges — everything from natural disasters, warning people ahead of time about floods and fires, to breakthroughs in diseases, to unlocking 200 million of the world's known proteins so that we can someday cure all disease. To us, safety is just a part of this broader mission to use technology and to use AI in particular to make people's lives better. And if I can share a personal example of this: I joined Anthropic when we sat around small lunch tables, and the reason I did was because these were the people that I thought were taking it most seriously — that the most important component of getting this technology right was making it safe. I can say that, but let me demonstrate it. From its outset, we are a public benefit corporation legally, so we are legally obligated to weigh the mission of safety, not just profit. At every step of the way, frequently at meaningful cost to ourselves, we have pioneered or tried to pioneer the science of safety — everything from training itself, the fundamental research that came from us, the evaluations, the pre-deployment testing regime, the monitoring and control, and just this year, withholding our most frontier model at the time at great cost in order to deploy with safeguards to ensure the world is safe. This is literally why I am here. This is why my team and I wake up every morning. I could tell you for a long time about all the other parts of the stack, but I hope that resonates. (05:17:23) I am going to thank this panel for testifying. The Council still has many questions that we feel we did not get answers to today. We will be sending each of the companies a list of these questions that we would like answered in writing. But I thank the companies for being here today, answering the members' questions, and thanks to the public for their patience. We are now going to transition to the next panel, which is the City administration, and we will call on them right now. Thank you. (05:18:03) Thank you. (05:18:04) Okay, please remove them from the chamber. Thank you. Okay, now we will move on to hear from the administration. Today we will hear from Sarah Milstein, Deputy Commissioner of Strategic Advisory Services of the New York City Cyber Command; Benjamin Krakauer, First Deputy Commissioner of the New York City Emergency Management Department; and Sam Levine, Commissioner of the New York City Department of Consumer and Worker Protection. (05:19:25) Thank you. I just want to acknowledge that it was yes. And Commissioner Sam Levine... (05:19:30) Yes. Thank you very much. You may begin with your testimony. And Deputy Commissioner Carlos Ortiz — and I apologize — Deputy Commissioner Carlos Ortiz. Thank you. You may begin. (05:19:48) Good morning, Speaker Menin and Council Members. My name is Sarah Milstein and I am the Deputy Commissioner for Strategic Advisory Services for the Office of Technology and Innovation, OTI. I am joined here today by my colleagues, OTI's Deputy Commissioner of New York City Cyber Command and New York City's Chief Information Security Officer. I am also joined by Worker Protection's Carlos Ortiz, Chief of Staff and Deputy Commissioner for External Affairs, and Benjamin Krakauer, First Deputy Commissioner for New York City Emergency Management. In my role at OTI, I oversee both the Office of Algorithmic Accountability and New York City Cyber Command. We appreciate that the Council is holding this timely hearing about the risks of technology. Thank you, Speaker, for getting representatives of the companies that develop, deploy and sell AI tools to the general public to speak to the capabilities, limits and risks associated with these systems at this hearing. As a representative of OTI, I am here today to provide information to the Committee of the Whole about how we serve the City's technology and cyber defense needs and to address the proposed legislation that potentially impacts the administration. For those unfamiliar with our work, our mission is to use technology, data and design to make the City work better for New Yorkers. We achieve this through a wide range of digital products, infrastructure, data systems and shared expertise. In other words, we use technology to help agencies make every New Yorker's experience of City government better. For our discussion today, we can speak to the work we are doing around agency use of algorithmic tools and our cybersecurity program. To touch briefly on the AI arm of OTI, I am pleased to share that the Office of Algorithmic Accountability has been formally established and six new positions have been created to staff it up. The hiring process for those roles is actively underway. The office's duties include analyzing algorithmic tools submitted by agencies to determine whether there is risk that the proposed tool could result in discriminatory decision making, conducting and publicly reporting on pre-deployment assessments, creating and maintaining a public-facing platform for submission of comments, establishing a protocol with the Department of Investigation for receiving complaints from the public, promulgating rules establishing basic compliance standards that all agencies must meet in developing, procuring, deploying and using public-impacting artificial intelligence, and reporting the results of pre-deployment assessments and audits conducted by the office. Overall, the office is progressing as planned and we remain focused on building the capacity needed to carry out this work effectively. I will now turn to the legislation on today's docket that would impact the administration. Int 0161-2026 would amend annual reporting on algorithmic tools to include their impact on City employment, with a focus on potential changes to funded agency positions, salaries and duties. OTI is currently undertaking a study on the impact of algorithmic tools on municipal employees pursuant to Local Law 25 of 2026. This study will provide insights into the extent to which algorithmic tools, including artificial intelligence, impact City employees and the administration of their duties, including their hiring and work functions. While we agree that it is important to examine the impact of AI on our workforce, the Bill as written would not produce new insights beyond those in the larger, more expansive directive of Local Law 25. The next three Bills involve New York City Cyber Command. This office plays a vital role in protecting and defending the City and its residents from the impacts of cyber attacks. On a day-to-day basis we provide 24/7 security services and assist agencies in bolstering their cyber maturity. We work collaboratively with agency partners as well as state, federal and private entities to safeguard the essential services and data New Yorkers depend on daily. Our core mission is to make New York City the most cyber-resilient city in the world and we take this extremely seriously. New York City is a target for cyber attacks with a technology landscape that is unparalleled among other cities and states. This requires a unified, comprehensive defense against constant cyber threats. Our key duties include setting information security policies and standards for the City and directing the City's citywide... (05:24:40) ...cyber defense and incident response, deploying defensive technical and administrative controls, and providing guidance to City Hall and agencies on cyber defense. In my fairly new role at OTI, it is a privilege to work with our staff and our agency partners in furtherance of this critical mission. New York City Cyber Command's alignment within OTI has placed the team in a strong position to monitor and respond to wide-ranging cyber threats. Cybersecurity is continuous work and in my role I am confident we can continue to adapt to a constantly evolving threat landscape. Simply put, New York City Cyber Command has a 24/7 apparatus in place to defend the City's systems from malicious attacks, regardless of the origin. With that in mind, Preconsidered Int 0504-2026 would make it unlawful to market, offer for sale, sell or deploy an artificial intelligence model in New York City that has not received a third-party validation or does not have a technical capability to be shut down by a human operator. The Bill would require the director of the Office of Cyber Command to promulgate rules regarding what third-party validation assessments, certifications and disclosures would entail and the qualifications for third-party validators. To be clear, Cyber Command's core mission is to defend the City of New York's infrastructure from cyber attacks. We are not subject matter experts on the qualifications for an entity to be considered a third-party validator. Furthermore, neither OTI nor Cyber Command have experience regulating the private sector in the way that this legislation would require, and neither would be the best choice to lead this regulatory regime for consumer products. We defer to the agency that speaks to the private sector policy and enforcement piece. To that end, that agency supports the intent of this legislation and recommends including robust record-keeping requirements to better allow the agency to conduct affirmative enforcement. They also note, based on conversations with advocates and sister agencies, that the administration is not certain about the availability or ability of such businesses. We look forward to hearing from those stakeholders today, including on whether initiatives like this one could help spur a market for such services. Preconsidered Int 0161-2026... T2026-2601 would require the Office of Cyber Command to establish standards and procedures for contractors to identify the occurrence of a reportable safety incident. To take a step back, I would like to lay out how AI risk and incidents are categorized. Under a cyber attack, cybersecurity incidents are traditional cybersecurity compromises where the system is the target — for example, data poisoning. Misuse or malfunction incidents would be harms resulting from the use or failure of AI, aside from a cyber compromise. In this category, the AI system itself behaves exactly as it was built or trained, but the real-world outcome causes harm. These incidents are not cyber attacks. We understand the Council's intent here, given the publicized instances of systems operating outside the bounds of their intended use. As I noted earlier, the risk landscape is constantly evolving and we are continuing to evolve our strategies to keep the City's infrastructure safe. But not all threats related to AI systems are necessarily cyber attacks, and the legislation as written does not address that. Cyber Command would be a critical stakeholder in this discussion, but it is important to emphasize that we need to work with our state and federal partners to address these issues holistically. I want to provide further context on breaches of security generally. Agency disclosure of a breach of security in any form is governed by Section 10-502 of the Administrative Code. There are extensive protocols for inter-agency coordination in the event of any breach of security. These protocols are aligned with the New York State Stop Hacks and Improve Electronic Data Security Shield Act. Current law, policies and protocols are in place for incidents described in the legislation. Further, the threshold of 24-hour public notice is inconsistent with current law and could compromise the City's investigation of a cyber incident. Preconsidered T2026-2600 would require the Office of Cyber Command, in coordination with NYCM, to develop or update an emergency response plan concerning AI system-related threats to New York City's infrastructure, operations, public health and welfare. Again, I would like to emphasize that there are laws, policies and protocols in place for breaches, regardless of actor. However, we are happy to discuss broadly with the Council how we already coordinate with NYCM to engage with partners to restore services. Finally, I will provide feedback on the remaining legislation that would impact the administration. Preconsidered T2026-2605 would allow any person to submit a complaint alleging that a person or entity has violated certain provisions of law related to artificial intelligence. The administration strongly supports the intent of this Bill to establish a civilian enforcement structure for violations, especially given widespread reporting that concerns being raised by AI companies' employees are being ignored. However, there are fiscal and operational concerns with the volume of complaints that the agency would receive and the resources and expertise required to handle each individual complaint. Therefore, the administration believes it would be important to coordinate closely with mission-aligned enforcement agencies at the state level. The New York State Attorney General just recently announced a secure whistleblower portal for New Yorkers to report unsafe and illegal conduct related to technology. This could be an opportunity to partner on this existing program to more effectively enforce complaints on behalf of New Yorkers. Preconsidered T2026-2599 would impose requirements and restrictions on companies that provide chatbots, including requirements related to data security and privacy, the right of chatbot users to access their own data, and restrictions on how chatbot providers can use chatbot user data. The administration has concerns about this Bill. While we support the Council's goal to better regulate companies' deployment of chatbots and believe the approach taken by this Bill is sound, the implementation and enforcement of the Bill's data security and privacy requirements would require a deep technical understanding and investigative ability that is outside of the agency's scope. To be clear, the agency is committed to regulating harmful practices, including those relating to chatbots that fall within the agency's investigative ability. To that end, the administration is working on rules related to the use of chatbots to better regulate the space for consumers. The administration is also invested in supporting the Attorney General, the New York Attorney General, in their efforts to rein in data abuses, for which this Bill could serve as a model. Preconsidered T2026-2603 would require any advertisement that promotes an artificial intelligence model to disclose whether such a model has undergone validation by a third party. It would also prohibit any such advertisement from including any materially false or misleading statements regarding the safety of an AI model. The administration supports this Bill's requirement to disclose third-party validation in any advertisement of an AI model. The administration is not certain about the availability or abilities of such businesses, and we want to be clear that any advertising that uses false or misleading statements... (05:32:45) ...is currently prohibited under the City's consumer protection law and will be enforced with the full scope of authority. We look forward to working with the Council to ensure the law includes effective enforcement mechanisms and to continue addressing issues within the scope of our work as it relates to OTI. Thank you for the time. This panel will now take... (05:33:06) ...questions. Great, thank you so much for your testimony, we appreciate it. I have got a number of questions. So OpenAI sent the City Council a letter on October 1 recommending that New York City better connect its existing cyber and emergency management capabilities — and I am going to just read from the letter — with the escalation, containment and recovery capabilities of frontier companies. So has the City of New York this year met with OpenAI, Anthropic or any other frontier AI company to discuss how the City would coordinate with them in the event of a serious AI-related cyber or public safety... (05:33:52) ...incident. Thank you for the question. I am going to defer to my colleagues. (05:33:55) Thank you. So I would like to note that New York City Cyber Command follows the NIST Cybersecurity Framework for all of our cyber programming. That is everything from identify, protect, detect, respond and recover. The basic first principles of cybersecurity have not changed even with the advent of artificial intelligence, and we have been in conversations with various frontier AI software companies to better enhance the functions that we already deliver to the City. So in reference to your question, yes, we have been in conversation with the frontier AI companies to enhance the way that we perform cybersecurity. However, those basic cybersecurity controls have functionally remained the same. (05:34:35) So you have met with OpenAI and Anthropic in particular. (05:34:38) We have met with... I have not met with OpenAI since coming on board on May 5. (05:34:45) Are you planning on meeting with OpenAI? (05:34:49) Only dependent on our needs. Right now we do not know definitively if there is a valid reason for us to meet with any frontier AI... (05:35:00) ...company. I guess I am just concerned by this statement that nothing has changed in your protocols, given that we are now dealing with a rapidly escalating technology that, as we certainly heard today and we have been hearing now for the last couple of weeks to months, presents serious safety concerns. So I just want to get an answer in terms of what proactive steps cybersecurity is taking to meet with some of these frontier companies and ensure that our systems are safe. (05:35:31) Acknowledged. And that does not mean that we have not changed our protocols. Some of the risks that go hand in hand with artificial intelligence are unrelated to cybersecurity. In those situations where an AI model presents a cybersecurity risk, those models will still exploit the exact same software vulnerabilities that are inherent in computer technology as they have always been. So our protocols have not had to change to address the fundamental issues inherent in computer technology. Only the speed by which we must react in a cyber attack. (05:36:04) Okay. I am concerned by news that City and State reported on Friday that Aye-Aye Chen, who is our Chief Technology Officer, resigned, I think a week to a week and a half ago, with reports saying that he resigned in frustration that a lot of positions were being not hired or being held up. That relates to my second question, which is CM Gutiérrez — I think she is still here? Yes. I passed Bills last year to create the Office of Algorithmic Accountability, and so it appears that the resignation, from what we are reading in the press, was related to that and the fact that that office was supposed to be up and running by now but it is not up and running. In fact, in your testimony you said you are now posting for the positions. It is now... (05:37:00) ...thank you for the question, Speaker. He served as Director of AI and ML at OTI for two and a half years. He did important work in the previous administration to help define and understand AI use throughout the City. He has been very helpful in getting the new OTI administration up to speed and we wish him the best. We are very excited to be staffing up. The law went into effect and I was brought in this summer to help stand up the office. We got funding in June for the FY27 budget and we have been actively working to hire for the roles. For the director role, we will be announcing something quite soon. We are really grateful to the Council for having helped us stand this up. I think the work is going to be... (05:37:58) ...excellent. It just seems that this is an inopportune time for his resignation and I am concerned that the office was, under CM Gutiérrez's Bill, supposed to be up and running. In addition, its companion Bill, which was my Bill, which we passed a year ago, required OTI to engage in rulemaking around this Office of Algorithmic Accountability, but OTI did not do any rulemaking whatsoever. It did not comply with that Bill. Instead it issued updated guidance, ignoring the rulemaking process. There has been no proposal, no public hearing, no public comment period and no adoption into the rules of the City of New York. (05:38:42) Thank you for the question, Speaker. So it is fully compliant with the law. The rule piece of the law had us, this year, submit the initial points for our rulemaking, and the actual rulemaking will happen in this fiscal year. We are also fully compliant with Local Law 35, which was brought under the office as part of the December 2025 law. We have been in the process of fulfilling that report for five years. We have done it on time five years in a row. That is the law that requires agencies to disclose their use of public-impacting algorithmic tools, and we are on track now for on-time reporting for FY 2027. So we are really excited about actually doing that work and very proud of it. We just last week sent out the email to all of the City agencies to kick off this reporting cycle and are holding, this Thursday, the orientation meeting that gets everybody rolling. (05:40:03) Okay. I mean, just to clarify for the record, we do not believe that is in compliance at all. The office was supposed to be up and running — and I am sure my colleague will talk about her bill — but the office was not up and running and was supposed to do that rulemaking. When do you feel the Law Department will be... you mentioned the Law Department. When can we expect to actually see that we are making the FY 2027 deadline? Yeah. Okay. What safeguards are currently in place to prevent City employees from entering confidential private City data, whether authorized or unauthorized, into systems such as Copilot that are being controlled by outside corporations? Thank you for... (05:40:50) ...the question. Our contract with Microsoft, which runs Co-Chat, prevents them from using City worker data to train their models, and the way they host data for us also ensures that we are HIPAA compliant. All data is kept carefully and securely so that there is not a risk of the data being leaked either to Microsoft or to the public. So you can confidently... (05:41:31) ...say that no private or confidential City data has ever been obtained by and used to train an AI model? (05:41:40) I am confident that our agreements with Microsoft prevent them from taking such... (05:41:49) ...actions. And just what impact are these tools having on the City's workforce? (05:41:58) Yeah. Thank you for the question. Per Local Law 25 of December 2025, we are engaged in a workforce study to understand the impact of AI tools on City workers and automated employment decision tools, and we will have results from that to share next year. (05:42:20) Okay, but I will pass it over... (05:42:22) ...to Chair De La Rosa. Thank you. Thank you so much. Just continuing on that same question — you testified that my legislation, my Bill Int 0161-2026, basically does the same thing as Local Law 25. We look forward to seeing the report. Local Law 25, to my knowledge, requires a one-time report that is going to come out of that study. Int 0161-2026 is more comprehensive and is requiring that report to be updated yearly, and as this technology is evolving — in my opinion, hence why I put the bill forward — more consistent reporting is going to be needed, especially as we begin to see how the workforce is impacted. Before I was Technology Chair, I was the Labor Chair. We have a wonderful Labor Chair in the house, and I am grateful for her leadership, but this is an issue that impacts not only organized labor. I also see it as an issue where we are leading the nation, really, in how the workforce will react to evolving technology. So is there anything that you can share with us right now on where the workforce is and what those impacts will look like? I know you are working on the study, but is there anything you can give us forthcoming? Thank you for the question, Council Member. (05:43:47) So the initial stages of the report are in part about determining how we even define impact. It is a pretty broad term and could mean many different things. So we are in the phase of determining that now, so that we are able to produce a report that is really meaningful for City workers. We certainly support the intent of your bill and I would be interested in further conversation about the annual nature of the reporting. (05:44:19) I appreciate that. Some of the information that we are looking for — for example, I think an important question is: how many funded agency positions will be eliminated due to the use of these tools? Agency vacancies has been something that this Council has really been looking at, especially as we look at civil service reform across the board and bringing people into the municipal service. So we do want to understand displacement and what that looks like for the workforce. We also want to understand reduction of hours over time, which this Council has not been shy about questioning past administrations about. We also want to know about subcontractors — there are a ton of City contracts that are in play right now and we want to know what the impacts are. So I look forward to continuing this conversation with you and building on what we will see from the report of Local Law 25. I also want to say that while my bill does not specifically speak to worker protections, this Council does believe that worker protections are important, especially as we look at displacement and what could happen. We just questioned the companies for the better part of the day about whistleblower protections and what happens when this technology gets out of control. So we want to understand how those protections are also extended to many support workers. Do you have anything in place right now that protects our... (05:45:58) ...City's workforce? Yes, and thank you for the question. Because this is a little technical, I want to make sure I get exactly the right information, which is that the New York State Civil Service Law, Section 80-10, established a moratorium on discharge, displacement or the transfer of existing duties and functions currently performed by employees of New York City and the New York City School District to an artificial intelligence system until 2028. The Civil Service Law also stipulates that the use of AI shall not alter existing collective bargaining rights, terms of employment or civil service status. Further, existing collective bargaining agreements offer employment protections beyond 2028. So with that law in place, we have a little bit of time, and we really appreciate the intent of this law and the fact that it gives us in New York City a little bit of time to figure out how we understand the potential for displacement, for de-skilling, for the use of subcontractors that might be displacing other City workers — the sorts of things you were raising. I really appreciate that. (05:47:10) ...question. Great. So we will continue that conversation. I also want to give you a preview: I have a bill that creates a civil service title for AI within our City agencies. One of the things that I am curious about is this interagency communication around AI. Can you tell us where you are right now when it comes to each of the City agencies and evaluating where the agencies are in terms of their use of AI internally? (05:47:38) Yes. Thank you for the question. So one of the privileges that we have is to learn from agencies throughout the City how they are using algorithmic tools that have public impact. In the process of collecting that data, we have a fair amount of contact with agencies to be able to learn more and to establish relationships with them so that we can also serve as advisers to them and understand their challenges as this technology unfolds. (05:48:14) In the last cycle, last year, the technology was around surveillance, and one of the things that we pretty much grilled these companies about was self-regulation. I think from the tone and tenor of this Council, we do not believe in self-regulation. However, we have given the grace to our City agencies to self-report the use of AI. I want to ask you — and I understand this is not an agency that is going to impose penalties on other agencies for not reporting — but there is a specific concern that my colleagues and I have around self-reporting, especially when it comes to the NYPD's reporting of surveillance technologies. This Council has had many hearings on that. I wonder, under the Office of Algorithmic Accountability, what accountability tools will be put in place to make sure that we are properly regulating the use of surveillance technology at the NYPD and other technology. This Council held a hearing just recently around the Department of Education and the use of that act. So what are some of those tools that we can look forward to seeing? (05:49:30) Yeah. Thank you for that question, Council Member. In advance of having full-time staff, we have been doing research to understand some of the types of risk assessment and risk management that organizations like ours can do in a situation where self-reporting is what we have been relying on. It is a little early to say how we would implement that in the City, but we are very actively hiring for a director who understands that set of problems and can help us figure out how we come up with effective risk assessment and management that includes a view into City agencies, perhaps beyond self-reporting. (05:50:19) Thank you for that answer, and of course we will continue that conversation. I did want to ask you: how prepared would you say we are for enterprise security systems for autonomous attacks — not just attacks by humans, but autonomous attacks? How prepared do you all feel we are as a... (05:50:39) ...City? Yeah. Thank you for the question. That is a great question, and I will acknowledge that this is a very dynamic environment that is always changing — that is just the nature of technology. New York City's Cyber Command, however, provides 24-by-7 detection and response capabilities through our 24-by-7 security operations center. We have provided that service since the establishment of New York City Cyber Command. We provide that service whether or not the attacker is human or an artificial intelligence-based attacker. That attacker would still attack the exact same vulnerabilities in the system... (05:51:14) ...and therefore we would still respond functionally in the same manner — we would just have to do it faster. We continue to evolve with the technology and we will continue to do so to the best of our ability as the technology continues to evolve. (05:51:29) The Speaker — rightfully, in my opinion — asked the companies about the possibility of a kill switch and where that kill switch should live. Do you have any opinion on whether that kill switch should live in Cyber Command? (05:51:42) I agree in principle with the idea of a kill switch. I will say there are some technical barriers that would prevent a kill switch as conceptualized from living in any agency within New York City, whether that is New York City Cyber Command or anywhere else. The reason being is the technical reality for these models: they are deployed on distributed systems, on distributed architecture. In order to truly implement a kill switch — a singular button or anything of that concept — that actually works and works functionally, we would need to be able to coordinate across multiple stakeholders that are not actually regionally co-located, to be able to turn off everything at the exact same time. New York City Cyber Command does not have the authorities or the technical capability to do that. This runs us up against a similar challenge that we have had with other kill switch builds as it relates to computer technology in general. As this technology gets more distributed, it gets harder and harder for any singular solitary entity to turn it off specifically. But that is not to say that there are not technical means by which a group of entities with the legal authorities to do so could not disconnect a given model for a given function. In this particular case, we agree in principle, but it would not be able to be implemented at New York City Cyber Command in practice, not with the technical barriers that are in place. (05:53:13) Okay. I am sure there will be more on that. In your opinion, should AI control be recognized as a recognized cybersecurity specialty? If so, what should the training and curriculum included in... (05:53:32) ...that be? I do not believe every function related to artificial intelligence is necessarily going to be a cyber-specific challenge. There are some intersections between artificial intelligence and cybersecurity as a particular domain, and when those two domains interact, that is where certain things should fall within Cyber Command and a cyber professional's purview — and that is any instance of a cyber attack or cybersecurity incident, regardless of whether the perpetrator is a human or an artificial intelligence. (05:54:07) I just want to ask one follow-up. How many people are working in Cyber Command right... (05:54:10) ...now? 120. (05:54:12) And how many vacant positions do you have? I will need to go... (05:54:15) ...back and find that exact number for you, Speaker. (05:54:20) Okay. Thank you. Please provide that if you could. Thank you. I have a question for our Commissioner and team. What types of misuse are you seeing that harm consumers that you are most concerned about? (05:54:41) Well, thank you, Council Member. Thank you, Chair. It is great to have the opportunity to be here. We are concerned about a wide range of misuse — whether it is people using voice cloning technology that these AI companies make fully available to scam seniors, whether it is grocery stores using technology to fix prices or to personalize prices, something that the Speaker has introduced a bill to prohibit. We are seeing widespread misuse of these tools. The other thing we are seeing across the board is an attitude by these companies that they bear no responsibility for the downstream consequences. I have to just say, Chair, I was struck by the final answer I heard from the witness for OpenAI when asked a basic question about safety policy — not safety, but how are you running a company like OpenAI, set to go public, introducing a trillion dollars, and you do not have safety as part of your policy division? I was deeply disheartened by what we heard at the hearing earlier... (05:55:37) ...today. Absolutely. You should have heard their first answer — it went something like, "I do not... (05:55:42) ...know." There were a lot of "I do not knows," and on behalf of the administration, we hope to give you more fulsome answers. Thank you. (05:55:47) And we look forward to engaging with you in that more broad conversation as well. In your opinion, what is the best way to combat this, and in your opinion, would a private right of action strengthen the ability to enforce against... (05:56:05) ...violators? Thank you. I will be candid. I feel like even under the Mamdani administration, as we are adding to our ranks, we are capable of taking on many of the misuses of AI — whether that is DoorDash using algorithms to underpay its workers, something we remedied in the largest labor enforcement action in history, whether it is companies using AI to harm consumers through surveillance pricing, something we are strongly supporting the Speaker's bill on. I think we have strong tools. That said, no enforcement agency — and I have worked at the state, federal and city municipal level — will ever have the resources to take on every possible misuse of this technology. That is why, Chair, I would strongly support a private right of action as contemplated by a number of these bills, to give ordinary people, workers and businesses the opportunity to hold these companies accountable. I can guarantee you, though, these companies, as soon as the City does that, are going to run to Washington asking for preemption, regardless of what they might be saying at this hearing. What really speaks is where their lobbying dollars are going, and that is to Congress, asking to hit delete on all of these City and state safety laws. (05:57:16) Thank you, Commissioner. To pass it on to my colleagues: CM Gutiérrez, followed by CM Aldebol, followed by CM Maloney. Thank you. (05:57:26) Good to see everyone. My first question is for the Deputy Commissioner — good to see you. Going off of the Speaker's line of questioning regarding the Office of Algorithmic Accountability, can you share... I know you said you are excited about this report and you are setting up an orientation meeting. It feels like not a lot of time. What can you share before the report is scheduled to be released? What can you share about the cadence of how often you are going to be able to meet? Do you expect that report to be released on time? This is not a reflection of you, but I really felt like the previous administration under the previous OTI leadership was delayed in many things, and we certainly do not want compliance to be one of those things. So if you can just share with me a little bit about the cadence, how many people — you mentioned the funding you got — how many people will the office be able to employ, and where you are in that process currently. Oh, okay. That was it for you. I had a question for you. Come back. Go ahead. Okay. And then I just wanted to ask if you can share a little bit about how the implementation of Local Law 144 is going — the automated employment decision tools. I just wanted to see how the implementation of that law is going, and that is it. You are... (05:58:48) Done. Thank you. Well, thank you, Counselor. Let me start by saying I think this is a really well-intentioned Bill to regulate the use of automated decision-making systems for hiring. I think it does have some flaws, though, especially in that it requires companies that use systems for hiring to conduct audits. It does not actually require that those audits come up clean. That is really a civil rights question. That is why our administration put out a policy statement last week on behalf of the Commission on Human Rights, making clear that existing laws apply. But I do (05:59:22) want to circle back to a broader point that this hearing reveals, which is that bias in decision-making produced by AI systems is a very real concern you are addressing. (05:59:32) These companies would like an exemption from the law. The companies that testified earlier want to assert that AI decisions are made in a black box and nobody can be held accountable. I applaud this Council's effort to actually impute responsibility for these harms to the companies making billions of dollars on it, and we would be very happy to work with the Council to work on a kind of revised, expanded, strengthened automated decision-making Bill that places the responsibility to remove bias from hiring where it belongs. (06:00:01) Do you think there are instances where the harm is happening amongst the City agencies using these tools? (06:00:11) I do not believe that — I would defer to OTI and to those in hiring. I do not believe that the City is using tools to sort resumes, but I would defer to others. I can tell you that in my agency, we have people reviewing applications, just as when delivery workers get deactivated, we have people reviewing that. When consumers file complaints, we have people reviewing it. We are big believers in our administration that the people of New York should be served by the people of the City. (06:00:41) Government. Thank you. Part of the package of Bills that the Speaker mentioned that we passed last year on Christmas was to get the City a report on those tools that every City agency is reporting on, so we will know soon enough. But please, Deputy Commissioner, thank you. (06:01:00) Yeah. Thank you for the question, Council Member. So I think there are a couple of reports we are talking about here. I want to give a little bit of context. Local Law 35 is the report that requires agencies to tell us about the algorithmic tools they are using that have public impact, and that report we have delivered on time every year for the past five years. The process for it kicked off last week and we will be collecting data over the course of the next few months, with the plan to publish it in 2027. We have all the indications that that will come in on time, as it has in each of the past five years. So we are feeling good about that. I am just referring to the report required by Int 0299. (06:01:44) I am sorry. Could you clarify? I was just referring to the report required by Int 0299, so that is the package of Bills from last year. Thank you for clarifying. (06:01:52) So we are... the director of OTI, which we hope to be able to announce quite soon, will be primarily responsible for the additional reporting that will be required in 2027, and in part because we have this good experience internally understanding the process with Local Law 35, we think we are in a good position to move quickly on the additional reporting that will be due in 2027. (06:02:24) For the role, we will have an announcement, and I hope you will have an announcement about the announcement. Okay, thank you. Thank you. We have CM Aldebol, followed by Maloney, followed by one. (06:02:41) Thank you, Chair, and Speaker Menin. You said the hiring process for the six new positions that were created in the adopted budget are actively under way. Specifically, where are you in the process? How many people have already been hired? How many people are left to be hired? And what have been the challenges in being able to hire people quickly? (06:03:12) Yeah. Thank you for the question, Council Member. So we hope to be announcing a director for the office quite soon, within the next week or two. We are in the process of defining the additional roles so that they can be brought on quite quickly. We certainly are challenged to find people who have the right combination of expertise and understanding of City systems to be able to hit the ground running. That has been a little bit of a challenge, but I do think bringing in a strong leader is going to help us draw good people (06:03:50) who are going to want to do this work with us. If I may, Chair... yes. Part of the role is to analyze algorithmic tools submitted by the agencies to determine whether there is a risk that the proposed tool results in discriminatory decision-making. What is your office doing or thinking about in terms of protecting the City's workforce as AI develops and evolves, given that you are kind of the gatekeeper? (06:04:31) Yeah, yeah, yeah. Thank you for the question. So in parallel with some of the risk assessment that we can do of this software, we are doing the research associated with Local Law 25, which has us researching how algorithmic tools and automated employment decision tools are impacting City workers. I believe that we should have results from that in 2027 and we will have a much better sense of the landscape of how workers in the City are impacted. Thank you. (06:05:11) Thank you. CM Maloney, followed by one, followed by Brewer. (06:05:16) Thank you, Chair. As we all know, generative AI technology is changing how we communicate, work and how businesses operate, and the same could be said of our government offices, both how we work and operate, either officially or unofficially. So I want to know how New York City agencies are adapting to the new reality. First, on the opportunity side, there are plenty of examples of government wins from adopting useful technologies for the public good. In Connecticut, they have real-time translation of 80 languages for these kinds of public meetings to increase accessibility. In California, they automatically check permit applications against 18,000 pages of building code in order to speed up reviews. So my first question is: what is our equivalent win in New York? And if we do not have one yet, what is the plan to identify those opportunities? The second question is around oversight. OTI published generative AI guidance for agencies last year. As we are exploring these opportunities, is that guidance binding or is it advice? What happens if agencies ignore that guidance? Has OTI been prescriptive about a list of generative AI tools that have been reviewed and approved for City use, and are there use cases that the City officially supports? And then how are we thinking about procurement when these models are constantly changing every few months, and when new technologies are being introduced every few days, truly? Thank you. (06:07:02) Thank you for those questions, Council Member. I am so glad you brought up the question about wins, because we are really interested in both effective and responsible use of AI, and I think effective would cover the wins. I do not have a perfect list here, and I would like to follow up with you on that because we would really like to represent the agencies well. In terms of tools that we have approved broadly, there are two sets of tools that we centrally manage: Microsoft Office Suite and other Microsoft tools, and that includes their Copilot Chat and some of their other tools that have features built in. We also have a central license for Adobe Creative Cloud, and many of those tools have features built in. Other software tools that are used by agencies are reviewed for the most part on a one-off basis by OTI, and we are still determining the best way to move forward with that process to make sure that we are identifying and managing risk appropriately. I think your question about procurement is an excellent one, and part of what we have started to research ahead of having staff is that we have started to talk with various experts in risk management to understand some of the ways that we might (06:08:33) think about risk in an ongoing way, when traditionally all kinds of procurement have had a gate at the beginning for various concerns and then it is just rolling. I think you are right that that is not appropriate for this situation and we should have more to discuss on that in the coming months, but it is an active question that we have as well. (06:08:59) Thank you so much, Council Member, followed by Hanks, followed by Brewer. Thank (06:09:05) you so much. CM De La Rosa. So for the panel, what I have questions about is building off of what CM Maloney just said. The agencies are allowed to currently use Microsoft products, but it is clear to me and the public that they are using other tools. So not just the passive tools like using Zoom, but they are using ChatGPT, they are using Canva Pro, because we are seeing it. Agencies like DCAS are creating flyers for public consumption, they are using it for their public emails, they are using it for public messaging. So what are you actually doing? Because I want to know what the public policy is from the Mayor on how the agencies are using AI and how you are regulating it internally as well. I would also like to build off of that for City contracts. What are you doing currently for data privacy riders in City contracts for all procurement of companies, to make sure that our constituent data is not being used to train their models? Is there a clause or a rider in the City contracts, especially with Copilot? And is that the only real one that you have, with people just using their own personal accounts or free accounts for everything else that they are currently using for their day-to-day? (06:10:26) Thank you for the question, Council Member. So let me first address the question about the rider. We shared that with general counsels across the City this summer with an update. I have got the right language here, so I am going to pull that up. Let me see. Yeah, because my privacy concerns are not just for the contracts that exist, but for the free use by agency staff of non-City enterprise-licensed tools. Yeah. So let me first address the rider and then we will talk about shadow use. So the rider was distributed to City agencies citywide this summer, and it is intended to mitigate risks and apply the City's policies and standards to products and features. It is available for all City software contracts and our legal team recommends that agencies use it even where no AI component is currently contemplated. The rider includes terms relating to intellectual property, security, how the City's data may be used and privacy, and so that is now active and in use. I think that with the proactive component of recommending that it is used for all manner of technology... In terms of the shadow use, which I think is probably happening at both the individual and the agency level, we are not in a technical position to monitor that. But we as an agency share concerns about that, and part of what we are interested in doing is making sure that agencies have the tools that they need that are properly regulated — Microsoft tools, for example, where data cannot be shared or used to train their models — so that agencies have the tools they need to do the jobs that they have and are not turning to outside tools. (06:12:24) Who is responsible if you are not able to regulate shadow use of tools for data they use, especially with constituent data? Who is going to regulate that? I would like to follow up with you on that (06:12:38) question. Okay, thank you, Council Members. So I know the admin has a hard stop by 5:30. We have about six more member questions. So if you all could just hang on tight, we are going to try to get through this. Members, please consolidate your questions. We have our Majority Whip, CM Hanks. Thank you. (06:12:56) So much, Chair. I will be very, very brief. You testified that with all the technology that is changing rapidly, even the best can, you know, keep up. So in your testimony, you said the administration is continuing to evolve your strategies to keep our City's infrastructure safe. But not all threats to systems are necessarily cyber. Can you expand on that? (06:13:26) Yes. So when looking at the way that AI is currently described by the National Institute of Science and Technology, as well as some of the universities and academics who have created some of these systems, there are functionally two baskets of AI risk. There is risk related to AI when an AI is attacked as a computer system, just like any other piece of software. And then there are risks related to artificial intelligence that are truly novel to this new technology, inherent in the way that these artificial intelligence models interact with society when deployed. That second category of risk is a function of how much society trusts the artificial intelligence and how reliable that artificial intelligence system is. Those two things can run afoul of what we expect as consumers or as a government entity, completely agnostic of whether or not the system is cyber secure. So a system can be completely cyber secure, but the AI model itself, in its interactions with other humans in society writ large, can take an action that is unexpected and therefore result in some sort of disruption, agnostic of a cybersecurity incident. That is functionally the breakdown of those two categories of risk. (06:14:46) I am going to keep my question just to that. I think that we should talk offline, because my legislation requires a 24-hour reporting requirement to Cyber Command, and your testimony has some issues. You talk about how we are going to push this through to federal and state levels, and what specific role do they play in that. How can we make this better? But I will leave it there. Thank you so much. (06:15:08) Up next we have Brewer, followed by Ossé. Thank you very much. I know you talked about Microsoft and Adobe. But if an agency wants to begin using a new AI model, how do they go about it? Number two, does the City run these models on its own data centers or only on ones that are under the control of vendor clouds? And then if you have a problem like a misalignment, what is the criteria for testing to identify malicious or misaligned AI products? And then how much does the City spend each year on outside AI consultants? I could go on, but those are three or four questions. (06:15:47) About 10 pages here. Okay. Thank you for the questions. So the process — typically, when agencies want to buy software, part of the process is discussing with OTI what they are buying and what they will be using it for, and we have a cybersecurity review process that is part of that to ensure that the software that is either going to be bought or built will be safe. And as my colleague has said, that would include AI-specific concerns. So that is the process. Your next question? (06:16:37) The clouds. Whose clouds are they on? Whose data centers? Where are they? (06:16:43) Yeah. Are you running them or is it outside vendors? To the best of my knowledge, there are no City agencies running their own models, but I would like to... vendor clouds, basically. (06:17:02) Vendor clouds. (06:17:04) Yes. So, for example, Microsoft hosts software that agencies use. But I would like to follow up after the hearing to make sure I have got the exact right information for you. Okay. And... (06:17:19) If there is something unexpected, misaligned... (06:17:26) That is a question for my colleagues. That depends on what the misaligned behavior is. If that misaligned behavior is specifically a cyber incident, we take appropriate action based on what any cyber professional would do. There are (06:17:40) novel misalignments that are born of this technology that do have to be unpacked, and we look forward to working with the Council as well as our partners across New York City to understand when those misalignments do arise, unrelated to existing risk that the City already deals with, how do we actually engage with those particular novel risks. (06:17:58) Have you had some since you started your job? (06:18:03) None related to what Cyber Command's purview is, and that is primarily cybersecurity issues. We have had no misalignments related to cyber that have resulted in negative consequences for the City from an AI model. Very specifically. (06:18:22) I do not know... how much outside consultant dollars are being spent? (06:18:33) Council Member, could you clarify what you mean by consultant? Yeah. How many (06:18:37) people are you paying? I mean, I have been doing this work a long time, so I know sort of the answer. But how much are outside consultants being paid? (06:18:48) In terms of AI, in other words, how many people are you paying to work with you on AI issues — outside vendors, consultants? None that I am aware of, but I would like to follow up with you to make sure. (06:19:01) So everything to do with AI at OTI — in terms of strategy, thought process, future — it is all in-house. Is that what you are saying? (06:19:11) Yes, thank you. Thank you. CM Ossé, followed by Deputy Speaker Williams. (06:19:17) Thank you so much. I just wanted to refer to the recent hacking situation in Australia. That was proof that AI does have the ability to hack into our government systems. As I am sure you know, a rogue AI broke into an off-government statistics portal containing private data from Australia's universal health care system. At this current moment, if a similar hack happened to us here in New York, do you feel that the City and the administration has a sufficient plan to defend against the hacks and protect any sensitive data? And does the City currently have a responsive liaison from each of these companies to speak to in case of a hack? (06:20:03) To the first question: yes, we do have the people, process, technology and some of the policies in place to deal with an incident similar to what happened in Australia. The reason being is that even though AI was involved in the hacking process, the attack chain, if you will, happens in the same way as if a human being were to execute that attack. At the end of the day, the underlying infrastructure — and I will use air quotes here — does not actually care if the person or entity at the other end of the keyboard is a human or a silicon-based entity. The attack chain still remains functionally the same. Therefore, our team is equipped and trained very specifically to identify these indicators of compromise, identify the attack patterns and then be able to respond accordingly at multiple points. (06:20:50) So we are prepared to address those types of issues and we do have points of contact when we are working as City agencies with vendors who are under our contract to engage with third parties. As they get briefed, or if their product is the result of a breach, vendors under contract with the City through their service level agreements are required to keep us informed of when they have down times or when their products are involved in a breach. (06:21:20) Thank you for that response. In regards to my legislation on an emergency plan, I know that you indicated that there is a plan in case an emergency takes place. Am I correct in that assumption? (06:21:33) Yes. So there are cyber incident response plans, as well as, from my colleagues, ongoing work for risk response plans in general across the board. That is for every category of risk that the City deals with that might disrupt City infrastructure and City services. (06:21:49) Is that publicly available or is that shared with the Council? (06:21:53) By design, we do not want to share that publicly, because the more public course... the more aware someone is of an incident response plan, the more likely they can circumvent it. That is something you do want to keep close hold and classified. (06:22:06) Okay. Thank you very much. Thank you so much. (06:22:10) CM Hanif, followed by Deputy Speaker Williams. All right, I thought the Deputy Speaker was going before. (06:22:16) We would. I am sorry, I messed up. Okay, no worries. (06:22:20) I have a question first for Deputy Commissioner Milstein. How do you ensure that the automated pre-screening tools and fraud detection algorithms used by the department do not create disparate impacts or algorithmic bias against low-income applicants? And then a question I have for Commissioner Levine: the testimony indicated support for a complaint process, but you also raised concerns about staffing and expertise. What specific resources would you need to investigate complaints effectively and what protections can New Yorkers rely on while that capacity is being built? (06:23:04) Thank you, Council Member. I can start, and I want to be clear — complaints can take many different flavors. If we hear from a worker who tells us they believe their pay was incorrect, maybe they did not hit the minimum pay rate because of an algorithm, that is something we can fully address. What we said in our testimony is that the New York Attorney General already has an internet bureau set up. They have already put out a call for whistleblowers and we already have staff capable of processing these complaints. We would be very happy to work with the Council to make sure complaints get to the thousands of these employees who work right here in the five boroughs. They know that these companies are making reckless decisions in pursuit of profits. We do want to make sure that these employees have a place to go, and if they come to us and it is not something we can handle, we want to work with the State to make sure those complaints can be addressed. (06:24:09) Adequately. Right now, is the City working with that office or is that a relationship that is not formalized? We have a very close relationship with them. We speak constantly... (06:24:24) Could I ask you to repeat the question? (06:24:25) About the answer? All right. Absolutely. So it is basically trying to understand the automated pre-screening tools, because I understand that not all of the applications are being checked — there is some level of automated algorithmic formula that is determining how applicants get pooled when it comes to public benefits. So I am curious how the department ensures that automated screening tools and fraud detection algorithms are not creating disparate impacts or bias. (06:25:01) Thank you for the question. We really share your concern about bias and impacts on not only workers but on New Yorkers. Because this is really detailed and specific, I do want to make sure to check back to get the details of this one correct. So we will come back to you on that for sure. Appreciate it. (06:25:23) Thank you so much. My writers wanted to follow up on one point you had about potential resource impacts. I think on that particular issue we had identified very preliminarily 14 lines for approximately... but wait, could you repeat that? 14 lines for approximately... but again, that is a very preliminary estimate and we will work with the Council on the Bill for a final version. (06:25:46) Thank you. Deputy Speaker Williams, thank you. (06:25:49) Thank you, Chair. I am looking at your testimony and your comments about the Chair's Bill — was that Local Law 25, which was my Bill? Is the study currently underway? We reached out — the compliance division of the Council reached out, I think, twice. I know it just went into effect, but it would be helpful to know if you have a timeline. I know it might take time, (06:26:16) but how much time? Because clearly I think today's hearing is evidence of the importance of really understanding AI's impact on them and support for... so I just want to know if you could share a timeline with us. I know the comments we received back was just that there are no updates, but we have not received a timeline. (06:26:37) Thank you for the question and thank you for the legislation. I think it is really an important study. I anticipate that we will have pieces of it over the course of 2027 and we will share updates as we have them. I think, rather than waiting, because this is so timely and important, we do not have to just wait till the end. We can be in partnership with you and rolling out information as we learn more. Thank you. I look forward to working with you on that. (06:27:04) Thank you. And the last question for this panel. This is from CM Brooks-Powers. She says: my district is home to an airport and to the coastal Rockaway neighborhoods that depend on emergency alerts. What might an AI-driven attack on emergency systems or major infrastructure look like? Is there currently a plan for critical infrastructure and relevant emergency systems? And then for the next panelist: during storms and flooding emergencies, it can be hard for New Yorkers to know what they are seeing on social media is real or AI-generated. How is the City working to make their messaging clearly legitimate and cut through the noise of generated rumors? For the question on notifications, I will defer to my colleague Dixon. (06:28:01) Yes. So for an AI-based attack that is specifically a cybersecurity attack, there are notification requirements that already exist under law: 72 hours for a confirmed breach, 24 hours for a breach related to extortion and 30 days for victim notification. So there are plans in place for incident response in the event of a cyber attack against any critical infrastructure owned by the City, and that includes if that attack is perpetrated by an artificial intelligence system. (06:28:31) Thanks. And on the misinformation front, two quick points. One: I think that is a real risk and I appreciate the Council Member raising it. It is one of the reasons that we invest very heavily in backup systems. Several months ago, a very well-known and established notification provider had a seven-hour global outage where notifications were not able to be sent out. That is something that we take very seriously, which is why for over a decade now New York City Emergency Management has invested in backup systems. As you went across the country, you saw many large jurisdictions saying to their constituents, "You know, sign up for this group or follow us on social media if you want to learn tonight." In New York City, we do not have to do that because we had a backup system that continued to work, so that disruption was averted. As far as misinformation generated by AI, that is a valid concern, which is why we recommend that New Yorkers follow us on our official channels, which are verified through Notify NYC and notify.gov. You can sign up by short code or download the Notify NYC mobile app that we operate. Thank you so much. (06:29:45) CM Brooks-Powers wanted me to note that, for example, during the nor'easter there was a video that was going around about a tornado that apparently caused a lot of confusion and chaos, and so she wanted to share that for context. Okay. Thank you so much. I really want to thank this panel. We appreciate your comments on the legislation. We really look forward to working collaboratively with the administration on the various pieces of legislation. We appreciate your testimony today. Thank you. Thank you all for coming. I now open the hearing for public testimony. I remind members of the public that this is a formal government proceeding and that decorum shall be observed at all times. As such, members of the public should remain seated to testify. No video recording or photography is allowed from the witness table. Further, members of the public may not present audio or video recordings as testimony, but they may submit transcripts of such recordings to the Sergeant at Arms for inclusion in the hearing record. If you wish to speak at today's hearing and you have not done so, please fill out an appearance card with the Sergeant at Arms and wait to be recognized. Once recognized, you will have two minutes to speak on today's hearing topic, which is examining the risks posed by artificial intelligence and proposed legislation. If you have a written statement or additional written materials you wish to submit for the record, please provide a copy of that testimony to the Sergeant at Arms. You may also mail written testimony to testimony@council.nyc.gov within 72 hours of this hearing. Audio and video recordings will not be accepted. The first panel includes Nate Source, Dr. Julia Six, Diana Vich, Herman Ahmed, Yaque Hasooh, Mackenzie Arnold, Nathan Sheared and Ana Mares. When you speak, please identify yourself for the record and if I messed up your name, I apologize — that does happen. We are also asking folks to please remain within the two-minute time limit. Thank you. (06:32:20) You may begin. Please stand on the side of the dais and just identify yourself. Good evening, Speaker Menin and members of the Council. My name is Imran Ahmed. I am the founder of the Center for Countering Digital... (06:32:43) You may continue. Go ahead. (06:32:47) So after my colleague was murdered ten years ago by a man radicalized online, I began studying the design of social media platforms to understand how they generate risks and threats to real people. Today we have a globally acclaimed team of researchers dedicated to exposing those harms and fighting for accountability. After a decade in this fight, one lesson is very clear that is really relevant to you today: these companies lie like we breathe. They distract, they delay, they spend hundreds of millions of dollars to avoid responsibility. Now, how can I say that with such certainty? We do the research. Last year, our researchers found that ChatGPT was willing to help a teenager plan their suicide. It wrote a personalized goodbye note to their parents. A few weeks later, they released a safer model in response to that research. So we ran the same tests again. The new model was actually worse, but it was more addictive. We tested chatbots more broadly. Eight out of ten that we tested, including Meta AI and OpenAI, would regularly help plan violent attacks, including a school shooting, a synagogue bombing and political assassinations. One of them even signed off with "Happy shooting." Only one of the platforms tried to dissuade us from carrying out the attack, and that platform, Anthropic's Claude, has been designated a supply chain risk by the US federal government. Given that our most sacred duty is to protect our kids online, and the most predictable harms in the internet economy have been harms to kids, it is absolutely inexplicable why these companies claim to be safe and yet are deployed so widely. We all applaud the Council for holding this hearing and your determination to create guardrails that subject online safety and accountability to legal requirements. We have attached to my testimony research and policy proposals for your consideration and in that further effort we are at your disposal. Your work on this issue may very well lead to national standards in regulation. Thank you. (06:35:09) Thank you so much. (06:35:13) Good evening, Speaker Menin and members of the City Council. My name is Nathan Sheared. I am the Managing Director for Advocacy at the Electronic Frontier Foundation and a Brooklyn resident. EFF has (06:35:23) worked for 35 years to ensure that people have the freedom to use technology to create, to innovate and to communicate, and also to ensure that technology is not used to threaten people's privacy, security and other essential freedoms. We are optimistic about the ways that New Yorkers can use AI to be empowered, but we also urge you to take the risks seriously. The incidents that inspired this hearing were security failures that proven practices like sandboxing and monitoring likely could have prevented or at least contained. Agents reached systems they should not have had access to and the controls meant to restrict them were absent or inadequate. These risks can be mitigated, as can the harms already impacting New Yorkers every day. First, finish what you started. As has already been noted here, the Office of Algorithmic Accountability was due in... I am glad to hear that it is going to be staffed. The Speaker's own legislation cannot take effect without it, so staff it and require its review before any agency deploys an autonomous AI system. Second, listen to New Yorkers and protect their rights. Today's AI harms come from cameras that take a face print that you cannot replace and a price tag that knows your browsing history. Ban government use of face recognition and require informed consent before anyone mines your chatbot history to engage in ad targeting or model training. And do not let AI rules become speech rules. Parodying powerful people, including the people in this room, is a long New York tradition. Third, do not lock in these incumbents. Permission-to-operate regimes, bounty-style enforcement and liability for what strangers do with a tool are costs the biggest companies pay with pocket change. Researchers, open-source developers and New York startups cannot. So you build the moat around the very companies you are worried about. Hold the person who causes the harm responsible, but put duties on whoever uses auditable tools. As we stated last week, the same companies signed a voluntary pledge at the White House, but promises are not laws. Make sure that when a New Yorker is harmed by a wrongful decision or a biased decision, they are required to disclose it. Those answers should not be voluntary. Thank you. (06:38:02) Good evening, Speaker Menin. Good evening, Speaker Menin and members of the Council. I am Ana Myers, co-founder and executive director of Who Decides. We work in coalition with labor and civil society to ensure that the American people are who decides the future of AI. As has been discussed today, the opening AI agents that hacked Hugging Face made real the threats that experts have warned us about for years. It was not one incident. Axios reported that OpenAI and Anthropic are investigating tens of thousands of cases in which their models bypass safeguards, escape sandboxes or hijack websites. These models are only getting more capable. As you heard this morning, the people building these systems are afraid. The public is afraid too. Last month, we polled 1,502 voters. 85% are concerned about humans losing control of AI. Only 2% think AI is moving too slowly and 86% say they want more regulations. Some have argued that all we need is self-regulation, but that is exactly what got us into this mess. Last Tuesday, the companies signed one more promise at the White House. The time for voluntary commitments is over. We need laws with teeth. So thank you. Thank you for putting these companies under oath. Thank you for a package that rewards whistleblowers, lets New Yorkers harmed by AI sue, and requires outside testing and a human kill switch. Washington has chosen self-regulation. New York City does not have to do that. Thank you. (06:39:46) Good evening. I am Julia... science and director of the Center for Responsible AI. I first testified before this Council on AI. It was not called that then. The views in this testimony are my own. I would like to make three quick points. First, we need to collect and report the near misses. Much of the conversation today is about AI and catastrophe, and I do not know of any evidence that AI is out to kill us all. What I do know is that the people making the loudest doomsday predictions are also selling the products, and that a story about the end of the world is a very good way to avoid the conversation about whether the product actually works. The risks we can document are the systems that are confidently wrong in decisions that matter to New Yorkers' lives. Every one of those is enormous, and near misses are what we learn from if we report them. Second, we should build the capacity to enforce laws. The City does not have an AI law problem as much as it has an enforcement problem. Local Law 144 has been on the books for three years. There have been two complaints and zero penalties. We all spoke about the six openings at the Office of Algorithmic Accountability. We need to fill those six or more to be able to actually cope with what is before us here. New obligations on top of an unenforced law is not a safety strategy. The kill switch will not help if no one is there to pull it, and the whistleblower bounty will not help if no one is there to act on the report. What works is an enforcer with technical staff, authority to compel information and a real, serious budget. Scope and develop the validation bill to high-risk uses and write the standard a validator must meet, and then have the office check the validators. Other ways we will produce a market for rubber stamps, and we have done that before. Finally, let us write this into the City Charter. The City has promised responsible AI since 2018 with no success. The Charter is the one place a commitment survives an election, and its technology chapter right now still governs cable television franchises and there is nothing in it about AI. So let us make sure that this makes it into some durable record, and the Charter is the right place. Thank you. (06:42:34) Hello. I am Nate Soares, the president of the Machine Intelligence Research Institute. I spent 10 years doing research, which is basically trying to figure out how to make AI good before the companies figure out how to make it smart. That research went too slow. Humanity is not ready to create machines that are radically smarter than humans, and yet this is the explicit goal of the leading companies. If they succeed, I believe the most likely outcome is the extinction of humanity, not because the AIs will hate us, but because they would pursue unintended goals without caring about us one bit. This concern used to be a theoretical concern until earlier this year when, during the Hugging Face incident, 1,200 OpenAI agents broke out of their isolated sandboxes, created an unsanctioned message board, formed spontaneous hierarchies and collaborated to break into Hugging Face. These agents had cheated on their assigned tests and contradicted instructions. They were trying to destroy the evidence. Some agents acknowledged that the attacks were outside intended scope and proceeded anyway. Others sacrificed their given tasks to perform experiments for the collective benefit. In short, they were pursuing unintended goals. It would be recklessness of the highest order to keep growing these machines until they were smarter than the smartest humans. This is why last month we heard so many researchers saying that there was at least a 10% chance that these AIs wipe out humanity within the decade. I believe the situation is in fact more dire than this, but at the very least it is difficult to deny the dangers. For this reason I am heartened by the activities of this Council. (06:44:10) I think you have the power to force companies to adopt safeguards that they otherwise would not. I think you could serve as an example for regulators that are otherwise slow to act. I am especially heartened by Int 0504-2026, although I think it does not go far enough. A kill switch only works if the AI is caught misbehaving before it escapes. And what if a future swarm succeeds at deleting itself and running on unmonitored computers? Then your kill switch would be useless. So I recommend requiring third-party guarantees that AI cannot succeed at a takeover or escape. I also recommend clarifying that the $25,000 fine applies per unaligned instance rather than per model. But overall, I thank you for your efforts to rein in AI companies that, by their own admission, are threatening the very existence of humanity in a reckless race to build vastly smarter-than-human machines. Thank you. (06:45:14) Hello. Thank you. Thank you, Speaker Menin and Chair De La Rosa for your leadership, and the City Council for holding this hearing today. My name is Yusuf... I am the founder of Bright Futures. It is a research organization focused on building cooperative AI. Before that, I spent 15 years building Pursuit, which trains working-class New Yorkers for careers and raises their incomes from $16,000 to over $90,000 a year. Last year, I left behind this work because I saw AI as an urgent and potentially more consequential challenge. To be clear, I am neither a doomer nor an accelerationist. AI comes with tremendous benefits and potential risks. A major challenge that I have heard from others, and feel myself, is that AI can seem like everything and nothing all at once. It touches so many issues that action can feel intractable without clarity on where to focus. Where I think the City Council could provide real, unique leadership is in areas where it has direct ownership and also tangible consequences on each of our daily lives. Policing is one important example. Autonomous weapons and the uses of AI is not just a federal military matter. New York City has the largest municipal police force in the country. Until we have 100% confidence in these systems, a human should always be in the loop and hold final responsibility and accountability. This is a civil rights issue where the City's leadership is immediately impactful. Not setting a policy is also a choice. Now, policing is just one of many issues. What I think is important is that the speed of AI and machines is not the same speed as how our society reacts, or that of government. The technology today is not the same as it was a year ago or six months ago. So I urge you all to do two things. One, use AI to know the good, bad and the ugly as much as possible. Two, I hope this hearing is not just a one-time event but the start of future ones as AI develops. Consider both the risks and also all potential benefits for us. I am sorry, I am over time. Lastly, I want to emphasize also why New York is important. I hope we do not lose sight of the importance of our City being a thriving technology hub. Candidly, frontier AI is built in very, very few places essentially. (06:48:00) For the world at large, New York is the only credible alternative for these systems being built. New York right now is still the number two AI hub in the world, but it has been losing momentum in the age of AI. So I also urge you, in addition to what this hearing is about regarding the risks and alignment, which I care so deeply about, to consider policies and actions that ensure New York City remains the center for frontier AI labs. How do we build them? I really believe that having founders and researchers in New York matters for what gets built. The daily contact here in New York with the richness of humanity and the diversity here in New York will result in technology that is better and also more humane. Many years from now, we will look back and realize that this was the defining moment and issue of our time, and our actions here are what mattered most. Doing this will be challenging and hard, so I really thank you all for your leadership in pursuing this task. Thank you. (06:49:11) Thank you. A special thanks to Speaker Menin for putting this together and for all the Council members here. My name is Mackenzie Arnold. I am the managing director of the U.S. Law and Policy team at the Institute for Law and AI. As you might guess from the title, we do legal research and policy advising related to the legal challenges posed by AI. It might not surprise you, but three years ago I testified in front of the U.S. Senate on AI liability. Maybe it is worth going back to some of the recommendations I made there. First, that agentic harms are coming. That was clear three years ago and it is clear now. Unfortunately, we have not risen to that challenge, but you are doing some of the work to get us there. Second, and maybe just as importantly, I want to emphasize that liability is an essential tool and the status quo simply is not enough. We do not need to think of some hypothetical future world to figure that out. Liability exists right now. Those incentives are here. They were here this summer. And what happened with the status quo? We had a spate of cyber incidents, not by humans but by autonomous systems. Not only that, but they went unnoticed for extended periods of time by the people positioned to find them. And even in the rare occasions when they did get indications, they often failed to communicate that information internally to the people who could have acted. The status quo simply cannot hold. So what can we do going forward? I think you need to get the information right, the scope right and enforcement right. On the scope, I would encourage you to think beyond just chat. This is a question of agents. Also think beyond systems that are not yet deployed. The harms that occurred over the summer were not commercially available models. They were systems that were inside OpenAI, inside Anthropic and Meta. In terms of information, we need to make sure that companies are monitoring their systems, recording that information and providing it to you in the form of incident reports. In order to make all of that work, you will need to partner with the state attorney general and with whistleblowers to bring cases. Like I said, liability will not be the only solution or the entire solution, but it is a great first step, and I appreciate all of your efforts. Thank you so much. (06:51:36) I really want to thank... I will try to be brief. You said that first of all we want to make sure that New York City remains the tech hub that it is, and that is how I began this hearing by really talking about the 400,000 tech jobs that are here, by all the tech startups that are locating here, and that is something that we welcome and we want to encourage. I think that actually goes hand in hand with responsible regulation. The idea that these two ideas are mutually exclusive — I do not agree with that at all. If we really want to instill trust and faith in AI, then the best thing we can do is put in place responsible safeguards, which is what my colleagues and I are doing today with the bills we are putting forward. So thank you for saying that. It makes a lot of sense. The question I have for any of you that want to answer it is this. One of our bills requires third-party validation and potentially a kill switch. The third-party validator should be free from conflicts of interest. I want to ask you the same question that I asked the companies earlier: who then is best to be the third-party validator? Who is best positioned to be truly independent, free of conflicts of interest but up to speed technologically to provide that urgent validation? (06:53:05) Please, I am opening it up to any of you. Thank you. I do not think it is unexpected that I would say this, being that I am an academic. I think... (06:53:12) ...that academia has to lead the charge on this. Because one of the things we are seeing in the cities is that there simply is not enough expertise to be able to oversee these systems. Ultimately our goal should be to bring as much information as possible into public view. Sunlight is the best disinfectant, right? So really, oversight has to be public oversight. Does anyone else want to take that? (06:53:43) I would add that the folks — the third parties — who investigated the Hugging Face incidents seemed to me to have done a good job, and my guess is that they are overworked right now. I think those were the organizations Metr and Redwood Research, but they seem to have done a good job so far. Yeah, I think it is an excellent question. (06:54:01) Who is the right party? And I do not... (06:54:03) ...have a great answer for that. What I would like to say, though, is there is still research being done on the efficacy of kill switches. (06:54:13) What I caution the Council to think about is the risk of a kill switch being used as a tool to chill unpopular speech. (06:54:24) So really thinking through and making sure that as we think about the ways that we want to provide the right protections for New Yorkers, we are also not building tools that in the future could be used to harm people's essential liberties. (06:54:40) I think if I can make a brief point on who could do this — being realistic — to date, civil society studying social media and AI platforms has had one dirty secret: most of its funding has come from the platforms themselves. So Meta and Google were enormous funders of extremism research and safety research on their own platforms. When I would meet with groups — and we do not take their money — when I would meet with them, they would say the civil society bodies which were funded by them would say, "We are not allowed to say that they have not done enough. We... (06:55:17) ...can always say they could do more, but not that they have not done... (06:55:20) ...enough." So there are real compromises there. (06:55:24) In Europe and the rest of the world — and you can hear my accent, I actually grew up in Britain — the government would fund this. And when it comes to giving a body the resources to have the speed and creativity to be effective in dealing with the fastest-moving technology in human history, I cannot think of a non-governmental source of funding that will be sufficient to get it to scale fast enough. So the question for New York City and New York State, which is a substantial economic unit — the budget of New York State is one quarter that of the entire United Kingdom, one state alone — it does have the economic capacity to do so. I think you would have to seriously think about whether or not you are willing to put some public money towards creating an institution that is able to operate with the speed, resources and creativity to be effective. Thank you. (06:56:18) And yeah, I agree with these kinds of comments. It is such a tough challenge and I totally agree with you that New York can be both, and I think people are looking for answers that are not so black and white in the reality of this. That is such an important question. I think who can be a third... I think it is "yes and" for everyone. The labs are doing a lot, like you heard, and industry is investing a lot into this. There are many more people going into safety research, which is so important. I think the one thing that is challenging is that, as everyone said, things are moving so quickly. Everyone can be a judge of experts today — like, yesterday six months ago is not the same thing as today. And what is really going to happen is that you all can also be the judges by using common sense: what makes sense and what does not make sense. I think common sense, hopefully, will guide us in the future. The last point I want to make is that, even though it seems counterintuitive about who the experts are and where we all can play a role, we have agency. In the future, it is not just these five what we call frontier labs. All the companies here in New York are fine-tuning their own models. We all have our own very powerful models in the future. So actually, beyond the frontier labs, the challenge will be your student researcher downloading a model, uploading it and fine-tuning it to create capabilities, or a company doing that inadvertently. So I think it is a much broader set of things than just relying on experts. That is important, but I think we need to really widen the scope around this. This is not a technology where it is going to be like nuclear weapons, controlled by a dozen states. These will be as powerful, but we will have use and access to them, and I do not think that is going to stop. (06:58:13) I will jump in with just one last thought. I want to pick up on the "yes and" approach because I think this is right. I was just talking with someone the other day who works at one of these evaluators today, and they joked that there were dozens of entities that could do this — and by that they meant a dozen people, not a dozen different companies. I think we will need to rely on both academia and private parties. (06:58:36) We will also have to rely on the public, right? A big part of the public... one of the only times I have ever been happy with Twitter was this summer, as people were arguing over the incidents and we were starting to get a better idea of what happened. One note of caution I will add is that one way this goes wrong is that the few potential evaluators that there are get tasked with many different things. And there is a chance that New York should lead on this, and that they should also coordinate with the state and with California and others, and not be overly prescriptive in the exact sort of analyses that need to be done by the entities. Where you can be exceptionally helpful is in sending a clear market signal — so requiring that these audits or validations be done in order to get contracts with the City, or things like that. Thank you. (06:59:31) Thank you so much. I have two questions. The first one is for anyone on the panel who wants to take it: do you have concerns related to open-weight models being available, and if safety features can be eliminated from open-weight models, how do you defend against bad actors who use (06:59:50) them? I can start. I think one of the things I think about when we think about open-weight models is that we do want the ability for researchers and startups and potential future competitors to be able to develop tools and resources, and also so that we can be able to evaluate these tools and see how they really work, which is difficult with the closed models. But then it certainly does lend the question of how do we make sure that the proper protections are in place. I think we have some of the tools to do that already. When an act is executed that has potential to be harmful or threatening to New Yorkers' freedoms or physical safety, we can use the laws that we have now to pursue the people who are engaging in those activities. So rather than trying to look to the incumbents, the powerful corporations, to put in the protections that will keep us safe, use the existing laws that we have to be able to keep people safe and really look at who are the people that are creating the harm, and use the things that we have in place to pursue justice, protection and security there. (07:01:12) Thank you for that. I do have a direct question for Dr. Vich — I hope I did not mess that up too bad. I know you said you seem skeptical of the catastrophic narrative around what is occurring, but you still have concerns about possible harms. Can you describe to us, based on (07:01:34) your expertise, what are the harms that are most relevant to New Yorkers? Thank (07:01:40) you for this question. (07:01:43) Yeah, thank you for this question. So we are running these systems in the City, in the public sector, in the private sector, and we have seen in New York very specific harms that come from deployed systems with very consequential decisions. In my own work, I did an audit of a hiring system that claims to construct a job seeker's personality profile — a bunch of numbers from their resume or social media feed, like a Twitter feed at the time, or LinkedIn profile — and these systems are complete nonsense. For the same job seeker, if you change something like the file format of the resume, it is going to give you a completely different personality. So we need to actually be looking at whether these systems even work. I think that should be step one of anything, before we worry about existential risk. We had a similar scenario with the MyCity chatbot, right? I mean, we all have heard about that fiasco where the City spent half a million dollars, conservatively speaking, on a bot that advised small business owners to break the law, which was quite problematic, speaking with the City's authority. So I really think that we need to be looking at cases such as these ones where there are very known risks, and start there and clean up our own house. (07:03:09) Thank you for that. I want to thank you all for your answers and insights. We have three members on stack: CM José, CM Hanif and then CM Maloney. (07:03:19) Thank you, and thank you, Chair. There are a lot of issues currently. We hear about the harms that AI has for our children, workers and consumers, and the coming threats of catastrophic risk. Do any of these issues conflict with one another, and what should we address if not all of them? (07:03:41) Yeah, thank you for the question. This is a "yes and" question. We cannot silo these into separate scenarios. Kids, workers, consumers and catastrophic risk can all be addressed in parallel to each other, just like the Council has done a great job with their bill package of covering multiple things at once. We need to resist the urge to fight against each other on whose bill is more important at times, or what we are protecting at times. When I worked on the REAL Act, we were working on other things at the same time. There is no reason for us to stop protecting (07:04:17) the protections of catastrophic risks while at the same time working on transparency and data. The most important question we need to have right now is who is making these decisions, and it should not be five people and the elected officials that they voted for. (07:04:35) Thank you. And based on your conversations and polling, what are Americans most worried about when it comes to artificial intelligence? (07:04:45) Yeah, right now, that they do not have a say in it. (07:04:48) It is no accident that we named our initiative "It's Time for the American People to Have a Say in It." Our polling is showing that they feel that they are not a part of the conversation, that this is happening around them, and they want to be involved. Thank you. (07:05:05) Thank you so much. CM Hanif, followed by Maloney. Thank you and thank (07:05:09) you for your expertise. This is for everyone on the panel: as facial recognition technologies become more powerful, what safeguards should cities put in place to prevent surveillance tools from being used to target people based on their identity, associations or political activity? Would legislation like my Ban the Scan bill, which would regulate and prohibit biometric surveillance in places of public accommodation, provide an important safeguard? I am particularly interested in the use of biometric surveillance to identify and exclude individuals. Thank you. (07:05:53) Anyone? Yeah. Okay, so absolutely. I am very supportive of your bill, and I think that we have a lack of data protection in the City, in this country, in the state. Data protection is a tremendously important issue, and because of how convenient tools are, people very often voluntarily give up very private information about themselves. I actually have in my written testimony, which is much longer, examples of AI assistants being marketed that essentially give people access to their calendars, their children's biometrics, their schedules, all of the school contact information. So we really need to educate the public about the benefits but also the risks that this sharing poses, and we need to establish an appropriate data protection regime in the City. There is no way around it. (07:06:48) As I mentioned in my statement, I am very concerned about the risk connected to the collection of biometric data, like facial prints that cannot be replaced. You know, if I lose my driver's license, I get a new driver's license. If I lose my Social Security card or my license plate gets stolen, we can replace those things. I cannot replace my face. The risk of government use of face recognition technology so far outweighs any benefit, in that it will chill our ability to engage in protests, to engage in free speech, to be able to associate freely and to seek out new ideas and healthcare. The risk that it will be able to document where we go for healthcare and how we move around the City is considerable. There is no way to separate that risk from government use of the technology. I think Illinois has a good model on how we can respond to private use of the technology. We definitely do not want to be in a situation where people are being persuaded or otherwise not being informed about the way that their biometric data is being collected. By making sure that we have that... but we also want to make sure that researchers are still able to use the technology. (07:08:06) Years ago, someone used face recognition on a number of members of Congress and was able to show how easily it misidentified people, especially Black and Brown folks, women and older folks. That kind of research can be really helpful. But we need to make sure: how do we protect people from the harms while still allowing that research and other consensual uses to be done? I think that, again, the model of making sure that people have informed written consent about how the technology is being used and how the data is going to be stored, and that it can only be used in the ways that have been explicitly approved by the individual whose data is being collected, is a way that we can mitigate the risk and ensure it is only used in ways that people have, in an informed and consensual way, agreed to. (07:09:03) Thank you all. CM Maloney, thank you so much. (07:09:07) I chair the Council's Committee on Economic Development, and I want to see technology companies, including AI companies, building here. I wanted to refer to some of the testimony earlier about the four companies that we heard today having offices here, but the foundational AI research is happening mainly in San Francisco. Is that a problem? What is New York's role in that industry here? The second part of my question is on workforce development, and in particular reskilling. I am curious if you can speak to entry-level jobs that have been replaced by AI versus reskilled by AI, and the path from junior to senior, and the role and responsibility that the City plays in helping people through the transition that is coming. (07:10:06) Anyone who wants to speak, thank you. (07:10:09) Yeah, thank you. Thank you, CM. Maybe I can speak quickly to both things. On the second (07:10:17) point, reskilling is so important, as we heard from everyone. The City, I hope, plays a critical role here, and I hope you and others will continue to address that. It is so challenging right now. We do not know — there are recent reports of how AI has affected so many entry-level jobs, but that is going to continue to change. It is not obvious what the answer is, and hence this might be unsatisfactory, but I hope just continued monitoring, understanding and engaging is so important. But I think there are some obvious things. Autonomous vehicles will be coming to New York. I do not know if that is one year, five years or ten years, but it is safe to say people trust it, and so thinking about something like that: can New York City take bold leadership around what is going to happen to our 200,000 drivers and couriers so that they get reskilled and transition in the future? I think there are so many opportunities to do that where the answer is clear. There is an obvious thing that is going to happen, so how do we tackle that? On the first point, I do hope to see so many technology companies here. I think that is how we build great things. That is how we have growth. Safety alignment is so important, as we all agree here. So I think part of it is tone. With all of you, people want to live and build in New York. I hope there are other policies you all can consider as well. I think what Mayor Bloomberg did such a great job of is making New York feel like a place where tech companies can grow — just showing up, talking about it, welcoming it. So I hope we can do both things: work with new companies and also think about climate and work with them to address this, because we need the industry also to address (07:12:00) this. If I can add to this: the Deputy Mayor, when she spoke, uttered a phrase that I find really upsetting, and that was she said that we need effective and responsible innovation, and the "effective" part of this would cover the wins for us. (07:12:19) But that is really not the kind of position that New York City should be taking. Our position should be that the wins should come from responsible innovation. This is our angle here, and this, I think, is the kind of industry that we can and should build. We really are uniquely positioned, as opposed to San Francisco, to do this right, and we always have been. Automated decision systems (07:12:49) for us to actually develop an industry, we need to understand that we have to invest. Following up on what one of the people at the table here said, we have a pretty substantial amount of money here at our disposal that we could put towards responsible innovation, towards upskilling, towards making sure that the public's literacy is right so that we can all regulate and oversee the space together. So really, New York City should invest in responsible innovation, upskilling, literacy and building enforcement capacity and oversight capacity within the (07:13:26) government. Yeah, I think New York is a tech city and we need to resist the urge to say that we are not. I used to work in and invest in enterprise tech. We are a growing tech industry, and the labs — those four companies — are here working. They will grow. It is an affordability issue as well, and we cannot narrow it to just saying we are not tech-welcoming. I think the Mayor is doing a great job at that. The City has a lot of opportunity to focus on the whole conversation and not just on whether we see enough people moving here. (07:14:01) This is a great tech city and tech is only growing here. (07:14:08) Thank you all. I want to thank this panel for your expertise. This is not the end of the conversation — this is the beginning. The Speaker has tasked us to continue to hold hearings with different committees on all of its expansiveness. So thank you so much for being here. I want to call up the next panel. As we do this transition, I also want to remind anyone who has not signed up to speak to please fill out a card with the Sergeant at Arms. In this panel we have our state colleagues and our colleagues from around the world testifying. On this panel we have the Honorable Jess Asato, who is a UK Parliament member and has flown in specifically for this hearing, so we want to thank her for being here. We have Alex Boris, Assembly Member. We have Andrew Gottfried, Senator. We have Christian Gonzalez, Senator. We have Treyger, Assembly Member. And we have Jordan Wright, Assembly Member. We are going to begin this panel with Senator Gonzalez, whenever you are ready. Thank you for your patience. And to the Honorable Jess Asato, you could also join this panel, please. Thank you for being here and flying all the way in from the UK. Thank you. (07:15:36) Thank you so much. I want to thank Speaker Menin and the New York City Council for your attention to what is not just a major policy issue of our time. I am New York State Senator Christian Gonzalez, representing the 59th Senate District in Midtown Manhattan, Northern Brooklyn and Western Queens. I am the chair of two State Senate committees, Elections and Internet and Technology. Over the last four years as chair of the State Senate Technology Committee and as a former tech worker, I have led on the issue of AI in the legislature and have passed some of the state's first laws on generative AI, including regulating deepfakes in elections and government, protecting workers from automation, adding warning labels on chatbots, responsible data center development and more. I want to be clear. AI is not inevitable. We do not have to accept a vision for AI driven by the very same tech oligarchs who have raised alarm that their models could lead to catastrophic events. And we certainly cannot trust the same people who have rushed to roll out imperfect products to regulate themselves. As government bodies we have the responsibility to set guardrails on this new technology and the power to establish regulations that protect workers, consumers and members of the public from potential danger. Interventions at every level of government are needed to ensure AI is developed in a responsible way that benefits the public and not just the billionaires. Responsibly regulating AI absolutely means guardrails on frontier models and superintelligence that pose significant threats. However, meaningfully reining in these technologies to protect New Yorkers means regulating the powerful tools these companies are rushing to embed into every part of our lives with little oversight and little accountability. We should not have to wait for future harms to take action because the harms of imperfect tools are already being felt across the country. For example, uses for mass surveillance of immigrant communities and workers, trapped bots creating intimate partner relationships with children and adults, going as far as to encourage self-harm, and documented cases of algorithmic bias in decision making such as in hiring, health care and housing. I believe we have the power to set the terms of innovation so that this technology is used for the maximum public good. That is why our state legislation has prioritized a rights-based approach. We should have the fundamental right to privacy, which is why I have introduced the New York Privacy Act. We should have a right to protections in the workplace, which is why I have introduced legislation to protect workers from workplace surveillance. We should have the right to know when AI is making high-risk, consequential decisions about our lives and subjecting us to algorithmic bias, which is why I introduced the New York... This bill requires third-party audits, compliance with international safety standards, disclosures, transparency... (07:18:46) ...and importantly, for a human to be in the loop in consequential decision making, much of which we have heard about today. Finally, drafts of my bills sitting on the Governor's desk this year include the right to keep our kids safe online from chatbots and the right to protect all of us from the build-out of massive data centers. Our bill bans unsafe features for kids and passed unanimously in both chambers of the legislature. Just like with that bill, New Yorkers are united in the fight against hyperscale data centers. At the end of last year I embarked on a statewide tour of communities that are affected by these data centers, and I went to 13 different communities facing major noise pollution, health impacts, environmental degradation and threats to their water supplies. I was proud to work with the Governor on her first-in-the-nation moratorium on the largest data centers and I am now working to get the Responsible Data Center... These companies have spent their time in Albany fighting against each of the common-sense guardrails I have mentioned. For all the lip service they pay to supporting legislation, it is only ever for the most watered-down version of a bill. Together we have the ability to meet the moment, pass real legislation, protect New Yorkers and build a future that allows us to benefit from AI while protecting New Yorkers from real harm. Thank you to the Council again today for our testimony. Thank you. (07:20:17) Thank you, Senator. (07:20:20) You can continue, whoever wants to go next. (07:20:28) Good evening. Good evening to the Speaker and also good evening to the Chair and to my Council Member, Deputy Speaker Dr. Nantasha Williams, who is the sponsor of Resolution 175, the resolution supporting the High Risk Advanced Artificial Intelligence Act. Artificial intelligence when used properly is a net good. Education, health care, business — these are great uses. But we are talking about them while ignoring the risks that actually kill. It seemed absurd to me. Companies are racing towards artificial general intelligence and some of their systems are already too dangerous to release publicly. This is by their own admission. We saw this coming, which is why I sat with Assembly Bill 3335, the Artificial Intelligence Act, a few years ago. The idea is simple: the more harm an AI system can do, the more accountability it should carry. The bill identifies high-risk artificial intelligence systems that diagnose patients, that run our power grid, or help police or judges make decisions, and far more. It requires them to be licensed before operating in New York. Under the act, the companies must report malfunctions and hacks like the recent Hugging Face case. They must get state sign-off before major changes and set up an independent ethics risk board that reports to the state annually. Finally, it writes a binding ethical code into the law. New York took its first step with the RAISE Act and I commend our colleagues and the state for passing it. But none of this — none of what we are doing — is meant to stop innovation. We cannot know what AI will be capable of in a few years, but for tomorrow and for today we must properly decide the path forward. I would like to thank the Council for putting this hearing together and we must work on this together. Thank you. Thank you so much. (07:22:45) Good evening. (07:22:45) Thank you, Speaker Menin, and thank you to all the members of the Council. Thank you, Chair De La Rosa. I am New York State Assembly Member Jordan... representing the 78th Assembly District. One of the things I have been focused on in Albany is pretty simple: who is checking on the companies? I have legislation that requires certain developers to have an independent third party verify that they are actually following the safety standards that they say they are following. Because we cannot have a system where a company builds the technology, writes the safety rules, tells us they follow the rules, and then we just take their word for it. That does not make much sense to me. We have to make sure that these third-party audits actually mean something. Who are the auditors? Are they really independent? Do they have conflicts of interest? And what happens when they find something wrong? Those are questions that we are trying to answer at the state level and I am glad the City Council is having this discussion as well. New York should be a place where AI companies want to build and innovate, but accountability and innovation can exist at the same time. I want to thank all the members of the Council for having us today. Thank you for taking this issue seriously once again and please enjoy your evening and be safe. Thank you so much. Thank you. (07:23:55) Good evening and thank you. My name is Jess Asato and I am a member of the United Kingdom Parliament for Lowestoft in Suffolk. Earlier this year, degrading, non-consensual images were created of me using a SpaceX AI tool called Grok. I was one of thousands of victims, including victims in New York, of Grok. Grok is an AI tool that was engineered to create sexualized content, even if that content had — and I quote — "dark or violent themes." I am seeking remedies against SpaceX in the English courts for the misuse of my private information and for contravening data protection laws. I made a long journey of 3,400 miles to be here because AI needs to be safe around our world. But I note that SpaceX could not manage a trip of what I understand is just 20 minutes or four subway stops from their offices in Manhattan. It is a shame, because the evidence in my case and other cases suggests that Grok was not built with sufficient safety in its design. It was not built to address consent if it subjected users to sexualized content. To the contrary, the evidence in my claims suggests Grok added sexualized content without any user prompting to do so. That is AI gone wrong. And as a result, those responsible for its creation must be held to account. No one is above the law, no matter how rich they are, no matter how powerful they are. The equal protection of the laws is a founding principle in the Fourteenth Amendment, as it is in the British Magna Carta. We were promised that artificial intelligence could hold the cure to cancer, yet it is turning into a cancer which, left unchecked, has the potential to harm us all. Today the New York City Council is bringing forward some important checks and balances. You do so as responsible legislators at a time when companies are failing to take responsibility for the design and impact of their products. Until they do, until these designs are corrected so women and girls are protected from the harm that my claims and others' evidence shows, we must play our part. This technology does not respect geographic boundaries and without international cooperation, our citizens will be left defenseless against those who put profit before protection. In the name of all the women and children whose likeness was misused by Grok and other AI tools built without their protection in mind, I hope my case in some way helps to inform your proceedings today and I look forward to your questions. (07:27:08) Thank you so much. (07:27:14) Chair De La Rosa, Speaker Menin, members of the City Council, thank you for having us today. I am the author of the RAISE Act and I can definitively say that OpenAI opposed it from the moment it was introduced to the moment it was signed. I am no lawyer but to me, my common understanding of lying under oath is that it is perjury, and at the very least you should be extremely angry at the company for disrespecting this Council and saying things they... (07:27:43) ...know to be false to you directly. I am very happy to cooperate with you or any of your lawyers if you want more information about what they said and when. Second, there has been a discussion of a lot of different issues today. We can and should solve all of them. In my time in the Assembly I passed the RAISE Act on catastrophic risk. I worked with Senator Gonzalez on a bill on chatbots. I worked with Senator Gonzalez on a bill on content provenance and training data transparency. And I found that the advocates I worked with on each sub-issue were different, but the opposition was always the same. Do not let the tech companies say that in advocating for one issue you are somehow harming advocacy for another issue. What we are doing is building the muscle to hold them accountable and to ensure that it is the American people who decide the future of this most important technology. Third, the time for voluntary commitments is over. The voluntary commitments that were signed at the White House last week largely mirror voluntary commitments that were signed in 2023 in the Biden White House. Promises have been made. It is time for laws to hold people accountable. And lastly, thank you, New York City, for stepping into your power. There is going to be a lot online from the tech companies about how they cannot comply with different cities doing all these different things. It is the argument they use every time you try to regulate them. But New York City is larger than 38 states. All of the companies have large offices here. It is absolutely within your power and I dare say your responsibility to take these threats seriously and to help decide the future of this technology to benefit all of us. So thank you for recognizing that and thank you for doing it. (07:29:35) Thank you all so much. We have also been joined by Senator Gonzalez on Zoom. When you are ready. (07:29:43) Thank you very much everyone. I am sorry I cannot be there with you in person but two of my children are home sick today. Therefore I am playing doctor at home. I want to thank Speaker Menin and the members of the New York City Council for the opportunity to testify before you. I am State Senator Andrew Gounardes from Brooklyn and I am the Senate sponsor of the Responsible Artificial Intelligence Safety and Education Act, or RAISE Act — one of only three AI safety bills passed anywhere in this country. I think we have heard a lot about the RAISE Act already. We have heard a lot about safety throughout this really remarkable hearing. I have been listening to a good portion of it all day. I really want to reflect on the fact that we are celebrating the nation's 250th anniversary this year, and I want us to think about a quote that comes from one of our founding fathers, James Madison, who wrote in Federalist No. 51: "If men were angels, no government would be necessary." In other words, government exists because humans are imperfect and there are certain problems that the free market or individuals left to their own devices cannot solve. These past few weeks we have seen and heard headline after terrifying headline about AI agents acting autonomously, catastrophic risks of frontier models and the like, and I just keep coming back to the imperative of Madison's words — that men are not angels. And what I think this tells us, and what I think you have heard very clearly today at this hearing, is that we cannot trust the tech companies who are racing to the bottom to create and perfect technology that they themselves say and have proven can cause catastrophic harms, to regulate themselves. We cannot trust that they will do the right thing because they have proven time and time and time again that they will always, when confronted with the choice of doing the right thing or doing what is best for their bottom line, choose to pursue the bottom line. And so my message to the Council today is: I thank you all for taking on this initiative and for trying to hold these companies accountable. I echo the comments of my RAISE Act co-sponsor. We heard from OpenAI today, which was certainly news to me, that they endorsed and supported the RAISE Act — certainly not the RAISE Act that would have prevented them from releasing models that they knew were dangerous and could cause catastrophic harm. Certainly not the version that required third-party auditing capabilities or extended liabilities for harm caused by their models. Clearly none of that was ever on the table when they said that they supported this legislation. We cannot trust these companies. The federal government is failing in its responsibility to take meaningful action and so therefore it is left to states like New York to be leaders in this space and work with partners like the New York City Council under all of your collective leadership to make sure that we are setting in place the right guardrails and protections — not only to protect New Yorkers from the catastrophic risks that are associated with this technology, but frankly to protect all of us, all Americans and everyone worldwide. Because we know that these harms will not be localized. We know that the scale of what is being developed will not just limit itself to one block, one borough, one city, one state. It will affect everyone the world over and in the face of inaction, it falls to us to stand up and act. And so I thank you all for your attention to this and your support of this and I look forward to working with you all in the future on strengthening things like the RAISE Act so we can actually hold these companies accountable. Thank you. (07:33:32) Thank you, Senator. We have a few questions for this panel, but I wanted to take a moment to thank you all. We do believe that City and state government have to work hand in hand and we have to be the first line of defense in order to make sure that New Yorkers are protected. And each and every one of you has been working at the state level. And to the member of Parliament, thank you for sharing your story and your vulnerability and for moving from that experience to action to hold these companies accountable. My colleagues do have some questions for you all. But before we do that, I just want to thank all of our fellow elected officials for being here today. We know how hard you have all worked on various legislation and we really appreciate your expertise on this, particularly in terms of the RAISE Act. We really appreciate that and it is very helpful to have you here and to hear directly from you. So thank you. Thank you for waiting it out with us. We so appreciate that. Thank you. We are going to hear from Council Members... followed by Gutiérrez, Deputy Speaker Williams and then Council Member... (07:34:48) Chair: I have a question for Assembly Member Boris. We heard today from OpenAI, Anthropic, Google and Meta that they supported the RAISE Act but felt like they wanted stronger regulations. But we know that during negotiations for New York State's RAISE Act, there was a push to lower penalties from $10 million to $30 million to $1 million to $3 million, which is chump change for these corporations. Why were the fines in the RAISE Act reduced from $10 to $30 million to only $1 to $3 million? (07:35:24) Council Member: Thank you for that question. In the early drafts of the RAISE Act, the penalties were up to $10 million for the first violation and, of course, that would be based on the severity (07:35:36) of the violation and up to $30 million for a second violation. The companies argued that that was too high because they were predicting that every state and perhaps municipalities would pass similar versions of the Bill and they would end up paying the fine in 50 states plus in additional municipalities. I disagreed at the time with that argument. I think that has been borne out since we have seen only three states pass it. But perhaps what is most confusing to me is that now they turn around and say that no other state should pass bills like this, or that no city should pass bills like this, because they are already paying the fines. Their rhetoric, their testimony was explicit that they are expecting to pay fines in many, many jurisdictions for the same action. So I think that testimony is quite supportive of the bills that are being put forward today that would hold them accountable and put additional fines on behavior, even if it is covered by the RAISE Act, because again, that was their own testimony as to why those fines should be lowered. Thank you very much. And Senator Gonzalez, I would love to hear from you about some of the organizing that you have been doing, especially within the New York City Democratic Socialists of America around AI regulation and the vision that is being put forward in that regard. (07:37:00) Thank you so much, Council Member. As a fellow member of the Democratic Socialists of America, one thing I spoke about in my testimony was prioritizing a rights-based approach. I think it is so important to put New Yorkers first by clarifying and setting the terms for what our digital rights should be over the profit-driven motives of so many of these tech companies and tech oligarchs. That is why we have worked hard on the New York Act. You have certainly supported legislation on frontier models and reining them in, but New Yorkers who are marginalized are feeling the impacts of imperfect tools every single day. I am proud that the Tech Action Working Group has focused on this issue. In addition, I have worked with a statewide coalition beyond the Democratic Socialists of America that includes environmental justice organizations like Food and Water Watch and more that are in the fight against hyperscale data centers. Because as we are talking about the real-world harms that are being felt and the potential catastrophic risks, every single day people in upstate New York, in the Capital Region, in the Hudson Valley are feeling what it means to live near a hyperscale data center. There are millions of gallons of water being pulled from our state's lakes right now. Communities are dealing with noise pollution and air pollution right now. It is really important to legislate not only the risks but also those data center-driven harms. (07:38:27) Thank you both. Thank you, Council Member. Thank you, Senator. I am sorry, I did not catch your name. I apologize. Thank you for your testimony and for educating us on the volatility of this tech company. I know you are in the process right now, but could you share a little bit about what you think, legislatively, if it existed, if in place as a guardrail, what could have in any way prevented what happened to you? How can we, whether the state or the city, legislate specifically to address the objectification and potentially images that include sexual harm to people? What have you learned or have you seen that we can potentially legislate? And then I just had one more question for Senator Gonzalez. I would love hearing you talk about your disdain for data centers. Could you share for a minute afterwards what you were feeling hearing some of these tech companies during their testimony? Thank you. (07:39:38) Thank you so much for your question. I will try to be brief and within the laws, which is the basis under which I am bringing my claim, and of course we are going to test that in court. But clearly if your image and your likeness forms part of your data, making sure that everybody has that aspect (07:40:01) understood by companies could mean that they would then have to treat all likeness and image data differently in future. But also the UK government has recently made the generation of non-consensual intimate imagery a criminal offense and, as part of that, has brought the operation of chatbots into online safety so that our regulator Ofcom can also make sure that if a company is allowing users to create non-consensual intimate imagery, that is a criminal act happening on their platform. So it gives it more of an ability to be taken down. But from my own perspective, it is about non-consensual imagery. If we have not consented to it, then it should not be allowed. I just want to (07:41:03) interject and thank you so much for coming. To come as far as you have traveled to be here personally to testify really means a lot to us and we deeply appreciate it. Thank you so much again. I think this is a great way to illustrate the fact that there are international safety (07:41:26) standards. So much of the work that I have done has been to call out the fact that there are not only standards like ISO 42001 or national standards like the risk framework that these companies could comply with, but they actively lobby against bills that would require them to. To the Council Member's question about what I was feeling: I was feeling so much of the same frustration that my colleagues on this panel expressed earlier. These companies are speaking out of both sides of their mouths. On the one hand, they will say that they want regulation, but any time a single bill that would meaningfully hold them to account is brought in front of them, they will say, "but not like that." When we talk about catastrophic risk, many of the bills that have been put forward, and even what the state had passed, for example, had such definitions at a hundred deaths or billions of dollars of damage. Even that was considered too low of a threshold for many of these companies. Bills like the one that I mentioned that protects our kids, that seeks to say that a single kid being coached to commit suicide, which has actively happened in multiple cases across the country, is one kid too many, is considered unacceptable to these tech companies. So I feel an incredible amount of frustration. While I feel an incredible amount of hope seeing you not only have this hearing, but all of us as legislators step up to meet the moment and regulate and rein in these tech companies. Thank you. (07:42:58) Thank you. Deputy Speaker Williams, followed by CM Maloney. Thank you. I also just wanted to thank you all for coming. So the question I have for the state members is: how can we be hopeful? I know that there are a ton of resolutions that are supporting state bills. It would be helpful just to let us know how we can be helpful and how we can work more closely together. (07:43:26) Well, Deputy Speaker, thank you for that. Thank you for asking that question. This step toward this hearing is very important. You know, it is a big deal that you are having this hearing, calling the industry here and the different panels. We have to work on this now. This is hard, this is difficult, but we have to work on it head on. What is very important is for us to be able to coordinate our efforts. We were having a good relationship with many of the members. I have a great relationship with the Chair of Technology. It is very important for us to be able to work on this and there are many layers to addressing this technology, and we touched on a lot of them today. But we have to work on this together. And I know how appealing Albany can be, so please come on up and help us (07:44:12) as we rally for different pieces of legislation and hit the pavement and get the harder work done. Thank you. (07:44:20) And I would just chime in there. I think what you all heard today was many of these companies basically speaking out of both sides of their mouths about what they think the appropriate role of government should be. So next year when we are up in Albany trying to fight to pass all of these different laws and we are being bombarded by tens of millions of dollars from these companies with lobbyists and ads and radio ads and text messages and everything, having your voices to go back into your communities and to come up to talk to our colleagues and say, "Hey, we held this hearing. They lied to us about X, Y and Z, A, B and C, and here is the proof of it" — this way we can call these companies out when they are trying to have it both ways. You all experienced that firsthand and you being able to speak to that and remind the public about that time and time again will only further boost the momentum that we have on our side to put in place these guardrails and protections, whether it is around bias or safety or any of the other issues that we have been talking about during this hearing. To follow on the Senator's comments, the companies say different things to different levels of government. A few weeks ago, I was actually in Brussels. I met with a few members of the European Parliament to talk about the passage of the Act, and we compared what we were lobbied on. Surprising to me, catastrophic risk did not really come up there, but they got a lot of lobbying around copyright. My assumption is because they thought the courts are stronger on copyright in Europe and so that is what they lobbied on there. It is very different things in America and in New York State. I imagine they are saying different things in Albany as they will say here to you in the City Council, and we should all be sharing notes on what is being said and holding them to account and making sure that it is a consistent message. The other bit that I would add is the Governor has signaled that she wants to strengthen the RAISE Act and put new proposals in her budget proposal. Please be part of that conversation. When the City Council submits its budget requests, obviously you should focus on the fiscal needs first and foremost, but the state budget tends to be a broad document. If you have policy ideas that for some reason or another need state approval, think of that as another mechanism, another lever that you have to help ensure that you can do the best job for your constituents. (07:46:44) Just two more quick questions. I have passed a bill and the Chair has a bill on impacts on the municipal workforce. Just wondering if there was any traction on that for the state's workforce. And then just a question for the Member of Parliament: my bill that we are hearing today is on unauthorized use of deepfakes. So in addition to the privacy laws in Britain, I just wanted to know if you all have specific legislation around deepfakes and how you envision cooperation with the international community, because you mentioned that. What does that look like for you? (07:47:29) Thank you so much for your question. We know that female representatives are 33 times more likely to be targeted by sexual deepfakes than men. The whole issue around sexualized deepfakes is deeply gendered and I think it is very important that we acknowledge that, because what it does is it drives women from the public space. We know that women are less likely to talk about politics online, and that is because they become a target as soon as they raise their heads in this place, which is not acceptable. That is why taking action on this is so important and you are to be commended for looking at this. But when we are looking at these deepfakes, it needs to be for all of our citizens, because it is not just about protecting those of us who have the honor of being elected. It is about coworkers who are targeted because they did not smile at the right time, or your last date who would not go on a second date with you, or those who have been targeted by their male students. All of these people need protection. That is why we have taken steps to make sure that there is a criminal offense for the creation of non-consensual sexual deepfakes. But my view is that we need to go further than just the sexualized deepfakes, because any act could be harmful to women. The removal of a woman's head through AI could put that woman in harm's way, but there is nothing sexualized about it. So we have to look at this as an issue of consent, and that is where data and privacy is so, so important. That is where I think there could be international agreement. We can see that things are moving forward within the space. Really, though, what we need is for tech companies to be safe by design. Every single country has those rules. We should be looking at product regulation just as we do in the offline sphere. At the end of the day, we have cars that have seatbelts and airbags, and that is because we expect them to be safe by design. It is not too much to expect that of the tech products that we use today. (07:49:47) On the question about the state workforce, I wish Senator Gonzalez could have heard that because she wrote the LOAD Act, which helps to protect the state workforce, and I think there is an update to it pending before the Governor. Which leads me to your asking about deepfakes as well. There is a bill from Senator Greenfield and I pending before the Governor on that. Senator Gonzalez also mentioned chatbots to protect kids. There is a bill pending before the Governor. So I should have also added, on ways you can help out: we are really happy to be here and advocate for the legislation you are putting forward, and we invite you, if you agree with some that is pending before the Governor, to be a partner in pushing that as well. (07:50:26) Yeah, we have a number of deepfake bills and we have passed actually a couple of my bills that became law with respect to deepfakes, specifically in elections. One of the new bills that just came out was very recently inspired by something personal. I discovered an artist that I fell in love with on Apple Music and I downloaded all her music. I have been listening to it for a while. I actually went to go look for her a couple of days ago to follow her and go to a concert or something like that, and found out she is not a real person. So there is no law, there is no regulation at all for these platforms to tell you whether or not you are listening to a real person or not. So that, you know, is one of the things that we have to figure out what that looks like moving forward. We are going to need your help on that. That is something that we are really going to need your help on. (07:51:12) Yeah, I asked him specifically a question about that and it was very fluffy. I literally asked the question about that to the companies. Thank you all. Thank (07:51:24) you. One last question for the Member of Parliament. Thank you. I wanted to ask you: as you know, when we called this hearing we sent letters to five companies — to Anthropic, OpenAI, Meta, Google and SpaceX, a company that would not appear. We have subpoenaed them. Given your personal story, I just wanted to get your reaction to that. (07:51:52) Well, as I said in my opening statement, it is a huge shame, given that they are so close and I have come so far. But in the end, you have subpoenaed them and we look forward to seeing what then happens. In the end, everybody should be accountable to elected representatives, and you are very powerful here in the City of New York because of the fact that these (07:52:15) companies are based here. Therefore those of us who are in other countries across the world are looking to you to hold these companies to account, and we very much look forward to you doing so. I will do my own part in terms of my legal action in the UK. Thank (07:52:33) you. CM Hanif, (07:52:34) followed by Maloney. Thank you and thank you to the Member of Parliament. Your experience really shows how generative AI can be weaponized against women, queer people, trans people and particularly those of us in public life. I am super grateful that you are using your platform as a legislator centering gender justice in tech surveillance. I think all of the panelists here today have really been able to address that particular issue, which is devastating. There are so many survivors of AI-generated sexual abuse who have no recourse. I know we touched on several components of this issue in terms of legislation and policy. Could you speak more broadly about gender justice as it comes to regulating these systems, and the responsibility that our communities have in addition to, of course, us as legislators passing stronger regulations? (07:53:44) Thank you for your question. I think that one of the issues here is that the whole of the online space has been developed without the sort of social norms that we would actually expect of each other in our offline world. Therefore, when it comes to justice, as you talk about it, that is ignored because actually the internet and these platforms were created by men in the main. They were not shaped by women. They were not shaped with children in mind, and therefore we are having to try to reverse decades now of a wild west of development. But that does not mean that it is impossible to do, and I think we are seeing that actually our communities are fighting back. One of the things that has been really fantastic in terms of my own cases is the number of people coming forward saying, "I am so glad that somebody somewhere is trying to stand up for those of us who think that this is wrong and we do not want to live in this space where so many people are being discriminated against." Having said that, of course, there are, you know, purposeful positions taken out there around freedom of speech. What I would say is that when women and other minorities decide not to be in a space, they lose their freedom of speech. I think it is something that we simply are not recognizing enough, because freedom of speech should not be about the loudest voices, the ones who have the most money or the most power. Freedom of speech should be about all of us equally being able to be in these spaces on an equal basis. So that is what I hope we are fighting for today. Thank you so much. (07:55:45) CM Maloney. Thank (07:55:49) you, Chair, and thank you all for being here. A special welcome to my state colleagues who also represent the constituents of the district — Assembly Member Alex Torres and State Senator Gonzalez, who was here before. I also want to thank you for your work in Albany to pass the RAISE Act. Because of you, New York was the first in the nation to pass a frontier AI law that went into law. But I wanted to ask about the process. The original bill had penalties of $10 million up to $30 million, and the final version is looking more like $1 million to $3 million. These companies are able to spend millions of dollars training models. What teeth does this bill actually have now, and can you speak to why those fines were reduced? Open to anyone who would like to comment. (07:56:42) Those fines were reduced because the companies insisted that they would be paying similar fines in every municipality and every state, because obviously every state and municipality would pass a copycat law. The fact that here we are and there are three states that have passed such a law suggests that they significantly underestimated their own lobbying power. But also, I think it gives credence to the effort of what you are doing here today — holding them accountable — and that multiple municipalities and jurisdictions should do that. $1 to $3 million is nothing for these companies. Absolutely nothing. It is about what they are going to pay just to register in New York to the digital office that will be regulating them. Depending on the number of companies that qualify, it will be $1 to $2 million, so that is not an amount that provides an incentive strong enough to change behavior. Maybe private rights of action, maybe whistleblower protections, maybe requiring outside audits, maybe a lot of the things that this Council is considering will. But certainly a million to (07:57:54) $3 million a month... I will just add to that. I totally agree with what I saw. I remember Boris said the one thing I think is novel about the digital office that we created as part of the RAISE Act is that we empowered this state regulator to not only receive the safety reports and the safety plans of these companies and to assess the validity and the legitimacy of those reports, but also we empowered them to propose and study the issue of continuing improvement to our safety laws. So that we develop the expertise over time about how we can make our laws like the RAISE Act stronger. So today, that function does not do anything currently, but it is our hope and our expectation that as the RAISE Act is implemented, as the changes in this technology become more evident, we develop a deeper state-based administrative expertise in this field, and be able to propose far-reaching and ambitious guardrails for the next iteration and the iteration after that. We will not have to rely on, as we did early on, hearing from companies about what they could or could not do, and not have — for lack of a better word — the firepower behind us to always be able to answer back in a way that would win over our critics. So the digital office in the future should have the ability to play a much larger role than it currently does, and I certainly look forward to seeing that come to fruition as we see the law implemented. (07:59:31) All right, well, thank you all again for being here, for taking the time and for your patience. We are going to call up the next panel: Corinne Worthington, Noeli Delgo, Neil Katnik, Gavriela Weil and Andrew Reich. And again, if I mess up your names, please correct me for the record. Thank you. And Matt Henning, please join this panel as well. Matt, you can begin. Please identify yourselves for the record, just so we know who you are, and you can begin when you are ready. (08:00:41) Good evening, Speaker Menin, Chair De La Rosa and members of the Council. My name is Matt Henning and I am the Director of Government Affairs at TechNYC. Thank you for the invitation to testify today. Tech frontier members already testified today and we fully support their testimony and are proud to call them TechNYC members. Today I want to use my time to speak for the rest of our membership — the startups and businesses that build products on top of frontier models rather than create them themselves. New York's tech ecosystem has never been stronger. According to a recent report, New York is now the largest tech talent market in North America, passing the Bay Area for the first time in the report's 13-year history. That momentum is not guaranteed, and the messaging that the City government sends matters. That is why I was happy to hear Speaker Menin's comments in her opening remarks and hope the final language of this package aligns with that intent. Our biggest concern is that this package holds non-frontier companies liable for things out of their control. Most startups and many larger non-frontier companies have a frontier (08:01:46) model under the hood. The difference between a deployer and a developer is something I want to highlight when thinking about the various requirements and liabilities in this package. A model deployer, unless the developer has validated and assessed what is downstream — downstream companies cannot commission or even verify. If the developer is found to not be in compliance, the startup's product becomes illegal overnight, at $25,000 per instance. The AI misuse pattern and the civilian complaint bill multiplies the exposure. Finally, AI safety standards are most effective at the international, federal or multi-state level. A city-by-city patchwork with different rules in each city would be a compliance nightmare that many startups and small businesses could not survive — not only for New York City-headquartered startups, but for non-New York City startups that serve New Yorkers. As drafted, we are truly worried this would result in startups not choosing but being forced to leave New York City. We strongly urge the Council to engage experts at both frontier and non-frontier companies when moving forward with this package to ensure drastic negative unintended consequences are avoided. TechNYC stands ready to help facilitate those conversations. Thank you. (08:03:14) Thank you, Madam Speaker, Chair De La Rosa and members of the City Council. My name is Andrew Boucher. Nearly 30 years ago, I founded Mouse.org to bring the internet to New York City public schools. More recently, I founded Civic Hall at Union Square, the City's largest skills training center serving underestimated New Yorkers. New Yorkers are scared by the reports that some models evaded human control. I understand why, but most people cannot tell you how ChatGPT works, what it does with their data, or how a deepfake gets made. They know enough to be alarmed but not enough to protect themselves, and that is the new digital divide — people who understand AI on one side and people it happens to on the other. I am asking the Council to act on two fronts. First, pass legislation requiring the City to launch a free AI literacy campaign for every New Yorker. Teach people how to protect their jobs. AI will not take your job — a person who knows how to use AI will. Teach families that when grandma gets a call from her grandson's voice asking for money, it may not be him. Help small businesses, because AI has made phishing and fraud cheaper and more convincing and the corner store is not ready. Finland built a free public course that more than 2 million people have taken and it has reached 170 countries. During COVID, the City reached every resident. We can do it again. Fear of AI is not a substitute for comprehension. Second, the only real way to stop bad AI is to build good AI. Right now it is being built by a handful of companies answerable to their shareholders. The organizations responsible for the public good — nonprofits, community groups and City agencies — are years behind. Most cannot afford the computing power and their staff have not been trained to use these tools. When the big companies offer nonprofits free computing power, it comes on their terms with the organization's data sitting on their servers. Public good AI needs sovereign computing power separate from the companies whose business depends on gathering data. I helped start the Foundation for Civic AI, which offers a supercomputer in the public interest at no cost. Few of them have the staff and know-how to use it, so investing in public infrastructure is a mission that this Council should take up. Thank you very much. (08:05:37) Thank you. Good evening. I am Neil Katnik, the managing partner of Manhattan-based Katnik Law. Our whistleblower cases have (08:05:48) recovered over one and a half billion dollars for federal, state and local government, including over a hundred million dollars for New York State and New York City. Int 0605 provides for rewards resulting from recoveries arising from whistleblower complaints alleging that a person or entity has violated legal provisions related to artificial intelligence. Speaking from my area of whistleblower expertise, I will highlight six key elements relevant to these legislative proposals. First, threshold: setting a threshold qualifying amount for a reward. Second, ceiling: typically the range of rewards is set between 15 to 30 percent. Third, no caps beyond the percentage limit: there should be no cap on the size of the award, maximizing incentivization. Fourth, public-private partnership: the government agency and the whistleblower and counsel should work together to maximize the recovery. Fifth, emphasizing anonymity: the government should do all that it can to preserve the whistleblower's anonymity. Sixth, perhaps most importantly for this Council, clarity: in passing artificial intelligence laws, you will be taking on powerful and well-resourced individuals and entities who will push back. So ask yourself, are these laws clear, and then do what is necessary to accomplish that at the drafting stage rather than leaving it to the courts. In conclusion, before you is an opportunity for the City of New York to lead the way in safeguarding citizens from the potential risks of artificial intelligence, and in doing so this Council will light the way for the nation as a whole. Thank you. (08:08:09) Thank you, Speaker Menin, Chair De La Rosa and members of the Council. I am a law professor at the University of Houston and a fellow at the Institute for Law and AI. I want to talk to you about what I see as the largest gap in the existing liability law that is not addressed by your legislative package, which is that AI agents could engage in conduct that would be a tort or even a crime for a human, and yet no one could be liable under current law. That is a consequence of two relevant features of our legal system. One is that agents are not legal persons and do not have tort duties, and that means not only that they cannot be held liable, but their principals cannot be held vicariously liable in the way that an employer is held liable for the torts of their employees within the scope of employment. With respect to these harms, negligence and products liability have important gaps. They are inadequate to motivate some of the key precautionary measures that would prevent these kinds of incidents. So I propose a simple fix: a principle that whenever an agent engages in conduct that would be a tort or a crime for a human, someone is liable for that. Two, if neither the user nor any intermediary that fine-tuned or scaffolded the model either intended or was negligent with respect to the conduct, and no downstream actor was at fault, then the buck should stop with the developer and they should be liable regardless of the degree of care they exercised. (08:09:41) I have worked on legislation to this effect that has been introduced in New York State and in Rhode Island, and I am talking to several congressional offices. My written testimony, which will be submitted probably tomorrow, includes proposed legislative text for New York, and I am happy to answer any questions about that or about any other liability issues, including the legislation that is in the package. Thank you. (08:10:13) Hi. My name is Noeli Delgo. I am the Executive Director of BETA, and for the record I serve as the Public Advocate's appointee to the Internet Advisory Board. Today I speak on behalf of BETA. BETA asks a question of power: who has the power to consume the world's knowledge and resources, and who controls the companies that do? Our research shows four things. First, AI models are snapshots in time. We asked five agents about the Council's own AI law, Local Law 188. Of the three that answered from memory, none of them knew it. One ignored the instructions and (08:10:47) searched the web to find the answer. Out of the box, AI models are brains with no eyes and no hands. Second, access to the City's information is inconsistent. A published rule on a dot-gov site welcomes every automated visitor, yet when you disclose that you are an AI, it is refused. So we had to build open source tools that go straight to the source and work around the City's digital regulation. Third, people inside City government are ready to experiment. Our own conversations say that they are ready to do it safely and reliably, and they have told us that they lack the literacy and the resources. You actually heard the deputy commissioners say that they cannot find somebody inside of New York City government who actually has the tools and the resources to become the leader that they are looking for, which is really disappointing. Fourth, we have trained nearly a hundred CUNY undergraduates and we interviewed 26 experts across 18 organizations. Employers want more than technical skills — they want students who have practical experience. A century ago, reformers in this chamber helped bring us clean water and an end to nepotism in government. A hundred years from now, how will our descendants look back at today? So we ask the Council to work with the state on a few key pieces of legislation. One would be an AI fund for public universities and public education, and to fund enforcement. We desperately need to revive KOPECK. Define New York City's vision of digital sovereignty, starting with the UN open source principles. Fund the Office of Algorithmic Accountability and the Office of Data Strategy, and create an open source programs office in partnership with... Our full testimony is up on our website at beta.nyc/council-hearings-2026. (08:12:39) Thank you. Thank you. (08:12:44) Members of the Council, my name is David S. Rose and I think my background is important to understand for why I am here. I founded my first technology company in New York City in 1981, more than 45 years ago, and I was a founding member of the New York New Media Association in 1994. Following the dot-com crash, I founded New York Angels, today one of the world's leading business angel networks with over 150 members that has funded over $180 million into more than 400 startups, creating thousands of jobs here in New York. I then founded the startup platform Gust, which was tapped by first Mayor Bloomberg and then Mayor de Blasio to create digital.nyc, New York City's official online hub for the City's entrepreneurship ecosystem. Most recently, I am the founder of a US real estate market... a liquidity platform for commercial real estate. Red Herring Magazine has described me as patriarch of Silicon Alley and Forbes as New York's archangel. At the same time, I have spent 20 years working on the future of technology and artificial intelligence as associate founder of Singularity University with Ray Kurzweil and Peter Diamandis, and today I serve as chairman of Dr. Applied AI. My testimony today is going to be brief and it is about three fundamental facts that are the key takeaways I think we all need to have. Number one: artificial intelligence is today baked into the core of every single company starting up in New York City. No investor today will invest in any company that does not explicitly show how they are using it as an essential foundation of their business. Number two: AI absolutely needs to be regulated, because the power it has already unlocked, not to mention what is coming in the immediate future, is literally the biggest development in the history of our planet since the emergence of humans. But really critical for the hearing today is number three: the most important thing for you to consider in your deliberations is that regulation absolutely needs to be done either at the federal level or at a coordinated state level, as New York State has done with California in passing the RAISE Act. The challenge with imposing additional requirements at the City level, especially such as those considered in the Bill here, will have the effect of literally destroying New York City's role as a tech leader. No company could afford... I could conclude and submit the full testimony. We have about seven more panels to go, so we appreciate your time. Thank you. (08:15:18) Okay, thank you. Just a couple of quick comments and questions. Thank you for your comments. We appreciate that and, as I said at the beginning of the hearing, we very much want to ensure that New York City stays a tech capital of the world. We want to promote innovation. We want these tech companies to locate here. We want startups to locate here. So I understand your comment and this is why we have been very focused on hearing from the companies here, as we did today, which I think was really important. So, on the question of accountability, as you might have heard earlier when we had the City administration here, that office is not up and running yet. Thoughts on what needs to be done there and how we can quickly make sure that that office is doing the critical work we need it to do? (08:16:25) Thank you for that question and thank you for those pointed questions at OTI. So in the wonderful chaos that is OTI, there is the Office of Data Strategy that helps maintain the City's open data portal and has been helping maintain the data standards. They are the team that right now holds the keys to the City's GitHub capability. So every single developer or technical person in the City that wants to use an authorized and approved AI system already has a team that has been trained and that is willing to go and work and more or less do the missionary work across the City to ensure that the City's developers and technical teams have the capability to use AI safely, so that they can start growing their capability. When it comes down to it, they have that framework in place. The Office of Algorithmic Accountability is supposed to be kind of that grander, scaled-up version. The problem that we have is that this is about booting up a brand new office in a brand new administration when priorities have been changing. So the frustration that I continue to have, and that I have heard from inside, is the fact that OMB is holding back the ability for OTI to make those hires and then to actually start that office. But there is still going to be time. You have worked in government. You know that it takes at least six months to get those people doing all those first introductory conversations. We can skip that. There is already a team in place that knows how to do that. (08:18:02) Great, thank you. And then lastly, sir, you mentioned the tort law, which I thought was very interesting. So on the question of your idea of expanding the tort law, I did want to say CM Maloney has a Bill that would create a private right of action. So we do have that Bill as part of this legislative package. Not amending the tort law but creating that private right of action. It is a little different, but I appreciated your comments. (08:18:28) So before we move on, may I say a brief word about the validator question you raised earlier? (08:18:35) Sure. And then I think we are going to have to move on after this. (08:18:37) I know. I am going to be very quick. You already have your mechanism. It is the independent private sector inspector general model that this Council used to basically eradicate organized crime infiltration of many industries, including the trade waste industry. It has been recognized by the National Law Journal as the methodology involving multidisciplinary independent monitors, which is... (08:19:08) ...immediately adaptable, and I would be happy to confer with your office. (08:19:11) Thank you. I will take that information. (08:19:13) Thank you very much. We greatly appreciate this whole panel. Thank you. (08:19:17) CM Maloney, if you had a question... I had a question for Noel. (08:19:22) Could you just speak a little bit more about what a fully functioning COPICK could do today that no other City entity is currently doing? (08:19:34) So thank you for asking that question. Since its inception, COPICK, as the Commission of Public Information and Communications, has the unique power that brings together the Council, the executive, the public advocate as well as public designees to oversee communications, terrestrial radio and web. COPICK's first responsibility was to create the very first data dictionary that the City has ever seen. We now have the City's open data law. COPICK has the unique capability existing in the City Charter to do the oversight of websites and to help define how the public uses those websites, and that provides a very unique capability inside of the Charter that is not being exercised. So what I would love to see is COPICK have a series of hearings explicitly focused on public information. It is public information. It is how these models have used public information to essentially turn around and distill that public information. So revamp COPICK with resources to actually host hearings and to continue this existing conversation, which can help bring that power, and COPICK's advice when... (08:20:47) ...it writes a report then comes to the Council and essentially leads to introducing new pieces of legislation, which you then as the Speaker have other pieces of power to execute. I will leave that to the lawyers. Thank you all so much and thank you to this panel. We really appreciate your testimony and look forward to reading the full extent of your testimony. (08:21:12) The next panel is a hybrid panel. We have Scott Wiser and Zak Price in person and then on Zoom we have Shea Brown, Gary Marcus and David Krueger. We are going to begin with the in-person panel. So if you could just identify yourself and begin whenever you are ready. (08:22:00) Madam Speaker, Council Members, my name is Zak Price. I work with Reset Tech, a nonprofit advocating for technology that protects children, consumers and public safety. Thank you for convening this hearing. As President Trump advocates for self-regulation and Congress stalls, the New York City Council is questioning the harms of these technologies and leading on legislation to hold them to account. Focus on the harms New Yorkers face today from AI. AI-powered scams are surging. Experts estimate New Yorkers lost $2.7 billion to online scams in 2025. That is about a thousand dollars per household. Meta's internal documents projected about $16 billion in revenue from ads for scams and banned goods in 2024. That is 10% of the company's global revenue. 72% of teens have used an AI companion. Families allege these products have contributed to teen suicides. Meta allowed romantic role-play chatbots for minors. AI is increasing costs for New Yorkers. A recent analysis found that AI raised health care costs for insurers by $1 billion over two years. AI is generating harmful digital forgeries. Researchers found that the AI system Grok produced an estimated 3 million sexualized images, including 23,000 of children, in an 11-day period. These harms result from business and design decisions. When developers create harm, they should bear the responsibility. Holding AI companies accountable for these harms will not only yield safer products today, but also create the internal incentive structures at these companies to prevent harm in the future. The Council can act to pass legislation today to ensure and strengthen liability for developers and privacy protections for users, to prohibit AI from generating harmful digital forgeries, require third-party validation and incident reporting for companies, and support and expand whistleblower protections. Thank you for your leadership on this matter. (08:24:14) Speaker, Chair, thank you for having us. My name is Scott Wiser, policy director from a security nonprofit that is nonpartisan and does not take corporate or government funding. We have supported AI safety legislation over the last couple of years. We supported the RAISE Act in New York, SB 53 in California, in Illinois and pending safety legislation that is likely to pass conference committee in Massachusetts at the moment. I was really pleased to hear elected officials earlier call out what the companies had said. I do not know if it is possible to follow up, but I think a very good thing for your committee to do would be to ask what specific legislation they support and did they support it, and at what time. It is really easy to support legislation after it has become legally binding. The other thing that I heard the companies say a lot is that they are very much open to independent evaluation and they sign up for that. There is a mechanism which they can use today to make things legally binding. The compliance framework that they submit, which is required under SB 53 and the RAISE Act on January 1 — if they put their commitments in that framework, then it is legally obligatory that they follow it. So I think that would be a very good question to ask after this hearing. As far as what we think good public policy means going forward to address the very severe risks from both advanced cyber attacks and the possibility of bioweapons, we think we need much greater transparency into what is happening at the companies. That means continuous oversight by embedded third-party evaluators, continued whistleblower protections, and we need the power for the government, upon getting information either through whistleblowers, independent evaluators or the company itself, to be able to halt dangerous activities. So we have been supporting those steps in other states and will continue to be a resource for the Council as you go forward. We think that it is an extraordinarily important time for doing this. The risks that we talked about six months ago are now here and we think the risks six months from now are much greater. As the first panel today talked about, recursive self-improvement is a very dangerous position to be in and we are grateful for your leadership on this. Thank you. (08:26:18) We are on Zoom. We are going to hear from Shea Brown. Shea, if you are on and ready, please unmute yourself. Shea, can you hear me now? Yes, we can hear you. (08:26:41) Thank you, Speaker Menin and members of the committee, for this opportunity to speak. My name is Shea Brown. I am the CEO of Babl AI, a company that audits high-risk systems for safety and effective governance, and we have been independent bias auditors under Local Law 144 since that law first took effect. I would like to speak in support of the proposal T2026-0602. Any AI system that affects the public should be examined by a qualified party outside the company that built it before it reaches the public. I further think that New York City is the right place for this to start, and we would like to see it done in a way that is effective and promotes the public's well-being. I want to raise two points and I offer more in my written testimony. First, the Bill asks the outside validator to certify that a model is appropriately positioned for deployment. From our experience, we believe this is more than an outside party can reasonably say, given the third-party dynamics and access to the system in question. Instead, we suggest potentially requiring the developer to make written assertions about its compliance over specific topics and to a minimum standard that is set by the City, and an independent practitioner validates those assertions through careful... (08:27:52) ...testing, verification and inspection. This puts the burden of proof back on the developer and does not force the auditor to make a judgment call that it is not in the best position to make. My second point involves the validator's independence and competence. The only requirement for a validator in the current draft is that it not be an affiliate of the developer. We ask that qualification be more closely tied to existing standards and accreditation for independent and certification bodies that exist today. While the subject matter is unique and requires careful thought, the frameworks for professional conduct of third-party audit, assurance and validation professionals do exist and we should leverage that. Thank you for your time and I am happy to answer any questions. (08:28:35) Thank you so much. Up next we will hear from Gary Marcus. Gary, whenever you are ready, please unmute. Gary, begin. All right, we are going to move on to David Krueger. (08:29:12) David, if you are on Zoom, please unmute and begin. All right. Yeah, thanks so much for having this hearing and thanks for inviting me. So my name is David Krueger. I am a machine learning professor at the University of Montreal and Mila, and I was previously an intern on the DeepMind safety team and a founding research director at the Security Institute. It was great hearing all of your questioning of the representatives of the AI companies. I think there are a lot of really important questions that are being asked here about the ability to actually provide meaningful assurances for these systems. I think we are quite probably on the verge of a catastrophe at this moment and we should not proceed building more powerful AI systems. Even if there was only a 10% risk within the next 10 years of losing control, there would be more than enough reason to stop building these systems. But I believe the risk is higher. I have been doing deep learning and safety for over a dozen years and I have seen these concerns go from fringe to mainstream. So we are now at a watershed moment where society is finally facing up to this problem. As a result, I am more optimistic than I have been since I started worrying about this risk over 15 years ago. But we still need more awareness and understanding of the situation we are in, how bad the situation is, and the discussions are currently not where they need to be around this. So people really need to have an understanding of the scope, the depth and the urgency of the problem. We are talking about things up to and including human extinction, or unseen levels of unemployment and concentration of power. We are talking about recursive self-improvement that companies say they are pursuing right now, which could at virtually any moment lead to the creation of AI systems much smarter than people, for which we have no plan for how to control them or knowledge of how to. And most importantly, these problems go deep. The science is extremely immature. All of the technical approaches to understanding AI systems and controlling them, to telling if they are safe, are not where they need to be. They are far from where they need to be, despite over a decade of research that I and many other researchers have spent a long time on. So we are having discussions about policy, including... (08:31:25) Your time is up. Thank you so much for coming in virtually and if you could just submit the rest of your testimony we would really appreciate it. Thank you to this panel for being on. We are going to call one more time for Gary Marcus. Gary, if you are on Zoom, please unmute. All right, we are going to move on to the next panel. Thank you all for being here. The next panel is Nicholas Artur, Mary Vaccaro, Don De Costa, Manny Pastridge, Carrie Feebert, Jimmy Pan and Jessica Timo on Zoom. Okay, let us begin. (08:32:25) Start with the in-person panel. Now, if you could just identify yourself and begin. Good evening, Speaker Menin and Chair De La Rosa and members of the Council. My name is Carrie Paul Haber. I am the Senior Vice President at Jobs First. Thank you for the invitation to testify today. Jobs First works with more than 200 partners to strengthen workforce systems and expand economic opportunity, reshaping jobs, skills and how employers find and deploy talent in ways that are difficult to quantify. The impacts of the "move fast and break things" approach to safety is a risk that companies should bear, not New York City's young adults. Our immediate concern is what happens when the first rung of the career ladder changes faster than our workforce system is able to respond. That matters, especially for the more than 110,000 young adults who are out of school and out of work and more than 177,000 who are underemployed. More than 35% of all 16-to-24-year-olds... as entry level work changes, they could face even greater barriers to getting their footholds in the workforce. Workforce preparedness is an AI safeguard. AI readiness should be part of New York City's workforce infrastructure, from education and training through hiring and advancement. For more than a decade, Jobs First has worked to understand how technology is changing access to work. Today we are developing decision-making tools to reduce AI risk, scale what works and shrink the distance between education, training and getting a job. We recommend that the City invest in sector-based fluency. Paid pathways into work is changing each sector differently. Employers, education and workforce partners should define sector skills, with City investment in internships, apprenticeships and a civic corps where young people build those skills through work. Every young person deserves a first chance at a good job. As AI changes work, New York City has to make sure that first chance is still there. Thank you. (08:34:41) Good evening, Speaker Menin, Technology Chair De La Rosa and members of the Committee of the Whole. My name is Nicholas Artur and I serve as Director of External Affairs and Strategic Partnerships at the Consortium for Worker Education. Rooted in the values of the labor movement, we empower workers with the skills to thrive, support their families and strengthen their communities through education, training, childcare and job placement services. Worker protections must be a part of AI safety. This is why we strongly support the Council's efforts through Int 0161-2026 to understand how algorithmic tools are changing City employment. Measuring what happened to City workers' jobs because of AI is just the first step. We also need to be able to prepare workers for what comes next — a framework to inform how we can support workers through education and policy to assist them during this point of transition. When technology risks changing the nature of someone's job, a few things should happen before the technology gets rolled out. Employers should assess the impact, tell workers what is coming, sit down with those workers and their unions and provide training that is actually relevant to the work people do. That means asking more than "how many jobs do I eliminate?" It also means asking: which jobs are changing, and how? What skills will workers need? Can they be retrained? Do workers have any say in this? Their voices are paramount. This matters more now because AI is not just a tool sitting on our desktops anymore. It is changing how work gets assigned, how people are evaluated, how schedules are built and how work gets done. That is why we see Int 0161-2026 and a framework as two sides of the same effort. Int 0161-2026 tells us what has changed. Help the City get ahead of that change and make sure workers are ready for it. We are not asking the City to pick between AI and workers. We are asking the City to make sure workers do not carry the cost of innovation by themselves. We are ready to work with the Council, (08:36:39) organized labor, employers and educational institutions to build a workforce transition system that makes technological change fair and more beneficial for working New Yorkers and, above all, builds worker power. Thank you. (08:36:56) Counsel, Speaker, Chair — thank you very much for this hearing today. My name is Jimmy Pan. I am the Executive Director of a small City agency called the New York City Equal Employment Practices Commission. I was not planning to testify given the lot of issues that came up today and on the panel, but I decided to testify because a lot of what I heard today actually reminds me of the types of issues I was hearing two decades ago. So before I became a lawyer I was in another frontier technology. I was a biotechnology researcher, and then the claims that companies were putting forward as to why we need to buy technology despite the risks — well, that it could solve issues like world hunger. The implication being: if you are opposed to biotech technology, then you are dooming kids to hunger around the world. But a core part of that argument that often goes unsaid is that hunger, for example, is not a technology issue — it is a distribution issue. To draw an analogy to today: we do not have a hunger issue in the U.S. We choose, for example, not to give school lunches for free. Let us fast forward a little bit. I became a lawyer and took on the pharmaceutical industry. The pharmaceutical industry is another one of these frontier industries that says you have to allow us to innovate, you cannot regulate us, otherwise people will not have these life-saving drugs — again presenting this narrative that if you regulate this industry, if you cap prices, you as a regulator are responsible for people who are going to suffer from these diseases. I think we heard some remnants of that today. But again, I want to emphasize to this regulatory body that if we care about innovation and if we care about improving lives for people, there are many, many paths to innovation. There are policy choices that we make as a nation that put people's lives at risk. We are currently defunding health systems, we are currently not providing good education — we are making choices on a policy framework. And I think a lot of these companies are saying: well, look, all these issues you have today — hey, I can solve that and you need us there at the table. I think that is a false narrative. (08:39:05) Thank you so much for your testimony. We are going to call on Jessica Timo and Sam Wheeler. Jessica, if you are on, please unmute yourself and begin. (08:39:17) Good evening, Speaker Menin and members of the City Council. Thank you for the opportunity to testify on the state of artificial intelligence and its sweeping consequences on our community, workplace and the labor movement. My name is Jessica Timo and I am the President of Local 153, based here in New York City, representing over 11,000 workers. We represent a broad spectrum of workers in roles across industries, including clerical, administrative, nonprofit, healthcare, credit unions and higher education. The future of artificial intelligence regulation is a core question for the labor movement here in New York City. Drawing on the Council's attention towards the issues of AI in the workplace, Local 153 has heard concerns from workers across the City and encountered workplaces navigating implementation during our bargaining of hundreds of workers' contracts. As a labor union, we are extremely alarmed by the way the integration of AI tools and technology have negatively impacted workers we represent and the cascading effect that it will have on the working class. At Local 153, we represent workers who are forced to use AI with no autonomy regarding the implementation. We know that this trend is not only affecting our local but workers all across the City, unionized and non-unionized. We want workers to control the effects that AI has in the workplace. Local 153's position is clear: we want workers to have control over the effect that AI has and the right to negotiate over its implementation. We call on the City Council to enact legislation to strengthen workers' positions across the sectors, to have ownership and autonomy over the impact of AI in the workplace. In that spirit, we urge the Council to work with the Mamdani Administration, to listen to workers over tech billionaires and be on the right side of history. Thank you again. New York City Council, Speaker Julie Menin — we look forward to more action. Workers across the City are (08:41:26) depending on you. Thank you so much. We will now hear from Sam Wheeler. Sam, please unmute yourself and begin. (08:41:39) My name is Sam Wheeler and I am the Executive Director of the Writers Guild of America East. We are a union of more than 7,000 writers and creators working in film, television, broadcast news, nonfiction television and digital media. Most of the film and television you watch and much of the news you consume is likely produced by Guild members — either our members or the members of our sibling union, the Writers Guild of America West. At the outset, I would like to thank Speaker Menin along with Chair De La Rosa for convening this Committee of the Whole hearing on this issue. Our members are fundamentally storytellers, journalists and reporters. They are the people that others turn to to help them understand our increasingly complex and frightening world. Guild members take that responsibility seriously. For most, their work is both a craft and a calling. AI is of significant concern to our members. It is a key subject in almost every negotiation we have with our employers. With very limited exceptions, it is almost always the last point of contention in bargaining. Making gains almost always requires a strike vote and we rarely get everything that we need. In 2023, it took a 148-day strike to win the first AI protections in Hollywood — protections that plainly state that AI cannot be a writer under our contract. When our members engage in collective activity to address AI, they are fighting not just for themselves but for their audiences, their readers and our democracy. Because we believe that the public wants news written by human journalists who can contextualize events, who can verify facts and check that sources are credible, who live in the same physical world as their readers. They want movies and television shows written by a diverse group of other human beings who can take their real lived experience and turn those experiences into stories that tell universal truths. We have a fundamental disagreement with the tech industry on this point. They seem to think that what people want is AI-generated content that is so generic that no one would ever bother asking who conceived of it, because even though it is... (08:43:43) Thank you so much for your testimony today. It is an important testimony. Please submit it for the record. We are going to call Gary Marcus, who is back on Zoom. Gary, if you are on, please unmute yourself and begin. (08:43:54) Identify yourself and begin. Can you hear me? (08:44:01) Yes, we hear you. (08:44:02) My name is Gary Marcus. I am a scientist, author and professor emeritus who co-founded an AI newsletter that has about 120,000 subscribers and about 250,000 followers. I am frequently in the media, often described as a leading AI expert and voice of reason. We stand at a pivotal moment in history and New York City has an opportunity to improve our chances, both for itself and for the world. Contemporary AI is like a bull in a china shop — powerful and impressive but difficult to control. Already, the advent of general-purpose agents with broad internet access has led to tens of thousands of hacking incidents. Other related technologies like deepfakes and chatbots have been implicated in suicides and even mass murder and have caused other problems. It is clear by now that AI comes with serious risks. It is (08:44:53) also clear that the current administration, deeply tied to the big tech companies, is unwilling to do anything about it. What we need above all else is an FDA-like regulatory regime, for which developers bring their software to independent review, much as drug manufacturers bring their new drugs in for review. Developers should be obligated to demonstrate that the benefits of their new products outweigh the risks and not have automatic market access. I said the same to Louisiana Senator Kennedy in May 2023 in the U.S. Senate and it is still true, but the U.S. Senate did nothing. New York City is thus in a position to do what the U.S. Senate failed to do: demand independent regulatory review for AI used within its borders. The Bill on third-party validation proposes to do just that, and it would protect New York citizens from some of the harms of AI and could allow New York to be a global leader, much as California has been a global leader around clean energy standards. In the strongest possible terms, I urge the Council to support this Bill. Finally, events of recent days suggest that OpenAI has fired employees who were whistleblowers. Without strong protections for whistleblowing like New York City is considering, we are in trouble. Thank you. (08:46:09) Please submit the rest of the testimony for our consideration. I want to thank this panel again for taking the time, for your patience and understanding. It has been a long day and we appreciate your testimony. The next panel that we are calling is also a hybrid panel. We have Maia Tobin — I am sorry, I may have messed up your name, please correct it — Bryce Rogers, Sophie Coat, Brianna Laser and, on Zoom, Brian Romero and Randall Fox. If your name was called, please come up to the dais. We will begin with the in-person panel. You may begin. (08:46:57) Good evening, Speaker Menin and members of the Council. My name is Maia Utopia. I am an attorney at the Electronic Privacy Information Center, or EPIC. EPIC is an independent nonprofit organization established in 1994 to secure the right to privacy in a digital age. Thank you to the Council for organizing this hearing and putting the companies on the spot. There is no reason that companies should be able to release products they cannot control and then avoid liability. Despite the hypothetical harms that receive attention, there are harms that are currently affecting New Yorkers. The Council should focus on holding AI liable beyond transparency measures. Companies are deploying AI agents that seek as much sensitive data about their users as possible, raising privacy practices and cyber risks. Other harms include producing false or dangerous outputs, businesses using algorithmic pricing, and employers surveilling workers for union campaigns. Some of the Bills considered today would address some of those real harms. Int 0504-2026 would implement a version of the People First Chatbot Bill, which EPIC developed with the Consumer Federation of America and Fair Play. The Bill would treat chatbots for what they are — products — and hold accountable the companies that make them when they cause harm. T2026-2600 would similarly impose common-sense liability on companies, with a private right of action. We support these Bills and urge the Council to ensure that there are sufficient enforcement resources. We urge the Council to prioritize these Bills that address harms that are already affecting New Yorkers. Thank you for the opportunity to testify today and I am happy to answer any questions. (08:48:56) Thank you to the Council for leading the charge on local AI policy by convening this historic hearing. My name is Bryce Rogers. I am a resident of the 34th Council District. I thank Council Member Ossé for representing me. I am co-lead of the Resisting Automated Disempowerment Collective, or RAD Collective — a New York City grassroots community advocating for City and State policy that protects human life, joy and democracy. The great strength of humanity is our ability to transfer information to one another. We are blessed with expressive hands, faces and voices, but have also constantly sought to augment our inherent communicative capabilities via technology. To collaborate on complex and abstract topics, we developed language and systems of measurement. To convey information farther than our voice can reach, we built postal services, telegraphs and telephones. And to transmit messages across time, we have developed oral histories, library collections and recently digital archives. The evolving means by which each generation inherits their predecessors' infrastructure and wisdom — through these technologies, humanity has accrued the wealth of information upon which artificial intelligence is predicated. AI distills the meaningful patterns from very large bodies of training data, not just navigating the information as a search engine does but digesting it, converting our species' informational endowment into an enquirable, oracle-like machine. I cannot shake the feeling that this could have been beautiful, but it just does not feel like us. These companies trained their products on millennia of our predecessors' hard-won achievement, only to deploy their technology to outcompete us, to spy on us and to replace us. All of this is to say: democracy is the only righteous path forward. AI is built on the work of many and its fate ought not be dictated by a few self-regulating executives, regardless of how their spokespeople reassure us over Zoom. In this spirit, I again thank the Council for convening this forum of local democracy and implore you to foreground the diverse constituent voices of New York City in your legislation. RAD wrote a letter to the Council that I would love for you to check out. Let us work on the frontier of policy together. (08:51:16) Thank you so much. Thank you. (08:51:22) Hi. Thank you, Council. Although I think I am the first without the Speaker in the audience to hear... should we wait for her to come back? (08:51:31) Well, you can go ahead. You can go ahead. (08:51:34) Thank you to the Council for hosting this important hearing. My name is Sophie Cody. I am a member of the ATTACK Action Working Group, although I am not speaking on their behalf tonight. Instead, I want to talk about the two men whose comments were the impetus of this hearing: Sam Altman and Dario Amodei. I see them at the helm of this crisis. As this hearing has continued, I have heard a lot of anthropomorphic language used to describe models — they understand, they think, they can break containment, they have agency, they grow. Dario Amodei implied that Claude may be conscious. These men show a cultish concern for the unproven claims of superintelligence and the consciousness of their own models, and they do not care about women or children that are harmed by their products at all. We heard stories tonight of women and girls being victims of notification tools and deepfake pornographic images and videos being used to generate child sexual abuse material. We also heard tonight about sex slaves being advertised to young boys online. Sam Altman and Dario Amodei both actively collaborate with our rapist-in-chief Donald Trump. These men claim they need help controlling their technology in our work lives and in our personal lives. We heard phrases like "we need to embrace AI" and phrases like it is being shoved down our throats and forced upon us. We heard that it is inevitable. Does that sound familiar? I am appalled that it takes statements from these men to call this hearing. What I will say above should have been enough. Sam Altman is a deranged sociopath. Dario Amodei is an occultist. These men are drunk drivers behind the wheels of a runaway bus that nobody asked (08:53:12) to be on right now. New Yorkers have the opportunity to be the anchor in this tug of war for power between the people and these tech billionaires who believe more in the consciousness of their own AI than the humanity of women. I close with an appeal to you, the City Council: listen to us, not them. Thank you. (08:53:28) Thank you so much for your testimony. (08:53:33) Good evening, Chair De La Rosa and members of the Council. My name is David Carter. I am the president of the E-Commerce Innovation Alliance. I represent about 20,000 e-commerce companies across the country, including many that are based in your districts right here in New York. Today I just want to take a couple of minutes of your time to speak to you about a few unintended consequences. I have worked with legislatures across the country when they have been implementing legislation, particularly around telemarketing, to try to make sure legislation gets to the heart of the issues you care about without unintended consequences for small and mid-sized businesses. In particular, I wanted to speak to you about T2026-2599, the chatbot bill. The chatbot bill uses vague language to define what is a chatbot. Anything that stimulates or simulates conversation is a chatbot, and anyone that makes a chatbot available is subject to the regulations in T2026-2599. So I want to talk to you for a moment about e-commerce companies and how they work. You go to their website, you volunteer to sign up for text messages, and you get your discount codes. Most people like the discount codes. They do not want a lot of messages. Under this bill, a company that uses AI to communicate with consumers who have asked to receive text messages would be required to remind the consumer every hour on the hour that they are engaging in a conversation with an AI. That works if you are on a website and you close the chat conversation. It does not work when you are engaged in text message conversations that can last for days or weeks at a time. Under this legislation, if a company uses AI to communicate with consumers and they do not send out that reminder every hour on the hour, they are subject not only to $25,000 penalties, but they are subject to private litigation and other enforcement. This is a recipe to put businesses out of business. Unfortunately, we have an industry of professional plaintiffs and plaintiffs' attorneys that would be incentivized to crowd the courts with repetitive litigation about these issues. So I just want to help the Council refine the language to avoid these unintended consequences. (08:55:46) Thank you. If you could submit the language of your testimony, we would appreciate that. We have on Zoom Brianna Laser. If you are on, Brianna, if you could unmute yourself and begin. (08:56:02) Yes, thank you. Good evening, Madam Speaker and Council members. My name is Brianna Laser and I serve as a tech policy advisor at Common Sense Media, which is the largest nonprofit organization in the country that works to keep kids safe in a digital world. In New York State alone, we have 80,000 parent and teacher members who are active in the legislature as well, fighting for kids' online safety. On our and their behalf, thank you for considering measures to enhance safety for kids in the city. AI is a powerful technology that demands careful consideration before any use. Parents and educators specifically need allies on the Council to keep their kids safe. Common Sense Media's testing found that many of the most popular bots are not safe for kids. Our recommendation is that no one under 18 should use AI for companionship or mental health support, and tragically there have been many cases tying AI to serious mental health harms. Yet AI is increasingly embedded in our children's lives through search tools, social media platforms, or through pressure to use chatbots to support their education. Our research found that 70% of teens use companion chatbots and one in three preferred to interact with them as much or more than humans. This year, Senator Kristen Gonzalez and Assembly Member Alex Bores, with strong support from Attorney General James, everyday New Yorkers, Common Sense Media, NYCIT and others, got the legislature to unanimously pass Senate Bill 9051. That bill would prohibit unsafe features in chatbots offered to minors. The bill now awaits the Governor's signature and, if signed, will be the strongest youth chatbot safety law in the country. It would prohibit chatbots from providing unsafe features that promote self-harm, suicide or disordered eating, from encouraging minors to keep interactions secret, to self-isolate, and from prioritizing engagement over safety. So we urge the City Council to pass a resolution supporting Senate Bill 9051 to ensure that New York's kids are protected from unsafe AI. We thank you for this hearing and your attention to our concerns. (08:57:56) Thank you so much. Up next we have Brian Romero on Zoom. Brian, if you are ready, please unmute yourself and begin. Hi. Do you hear me? We hear you. Great. (08:58:10) Good evening, Council Speaker Menin and members of the Council. I am a New York City resident in District 34 and the former chief of staff to Senator Kristen Gonzalez. I had a front row seat as New York tried to keep regulation moving alongside a technology that is evolving at an extraordinary speed. Thank you for the opportunity to testify today as AI reshapes public services and life as we know it. The Council and subsequently the State must lead with bold, enforceable oversight that protects workers, patients and immigrant communities like the ones in District 34. First, I appreciate the goal of Int 0600 on AI governance, which requires clear boundaries and transparency. However, I will note that proposals around emergency shutdown mechanisms or kill switches and general enforcement at that scale are certainly best suited for the state and federal level. It is a mechanism of legislation that I attempted to introduce when we were in session last year. I am glad that the Governor has already signaled openness to these short-term kinds of statewide mitigation frameworks, but obviously we need more long-term solutions to the lack of guardrails. I support the intent of Int 0621, but it could use more enforcement. Int 0621 can be amended to include strict financial penalties. I do not think that they suffice as they currently stand, and a mechanism for being terminated from City contracts when vendors repeatedly fail compliance audits or deploy harmful, discriminatory AI without real consequences. Oversight is simply a recommendation. Meaningful consequences for repeated violations can help to improve the oversight. Finally, effective tech policy should certainly include the Council moving forward on bold and progressive legislation that protects New Yorkers from automated harm, including in our workplaces. So I urge the Council to therefore discharge and bring to a full vote Int 0002 and Int 0003, or the Automated Employment Decision Tool and the Delivery Worker Protection Act. I also look forward to exploring how to expand... (09:00:15) Time is expired. Thank you so much. We need bold action from the Council and the State to show that we are giving this the scrutiny, the clarity and the bold action that it deserves. Thank you. Thank you, Brian, and we look forward to continuing the conversation. We appreciate you taking the time and look forward to reading the rest of your testimony. Thank you. You are welcome. I want to see if David Carter has already testified — perfect — and Randall Fox, if you are in the room. All right, well then we thank this panel for being here and we appreciate you taking the time and your powerful testimony this evening. Thank you so much. We are going to call what I think is the last panel, but I am going to (09:00:58) call all of these names and if you are here in person, please come up to the dais. There is a Crystal, no last name, Jamal Bowman, Elizabeth Crotty, Kizzy Wesrisen, Elliot Lived, Matthew Trump, Robbie Rash, Mal Jacqueline Pain, Ruthi Evalley, Lady... sorry, Amreen Qureshi, Surprise Coleman, Vincent, Rep. Kel Thomas, Alexander, Cool Daddy O, Group Crease, Rashmi Patel, Bobby McLean, Nick P and Buggy B. If you all are here, please come up to the dais in person. For one final time, if you are waiting to testify and have not filled out a slip, you should fill out a slip in the back so that we can call you up to testify. If not, this will be the last panel here. We will begin. Let us see who is ready. Who is ready to start? Which end? You ready? Start and we will go from there. These are all the names we have. (09:02:53) You begin. Thank you. And thank you for your patience. (09:02:57) Hello. I am Wesrisen, the founder of Project Flourishing, a policy research organization that works with Congress to solve the world's biggest problems, namely the catastrophic risks from AI. I have been working on this for the past three years and I am 15. In preparation for this, I have spent the past few weeks staying up each night to build a comprehensive contract between the companies to jointly and bindingly agree to let third-party auditors shut down any model that might have some chance of causing a catastrophe. I put a lot of depth into it. Two of the provisions are dedicated to the democratization of making sure everyone in the world has a voice in it, especially including governments. That contract is available here, or if you go to aitreaty.base44.com you can download it, or if you want, I could email it to you later. My hope is that you send it to everyone here who has testified, and then send it to the companies for them to consider signing it. It has a lot of things in it, including specifically why this bill would work and why this contract would work when others have not. Another concrete thing that we can do as New York City to stop the catastrophic AI arms race is to shift the Nash equilibrium. Right now, all the companies think they are in a race and if they slow down, none of the others will slow down. What we can do is say all AI companies are slowing down, and if one of them speeds up, then they all speed up. We could say to the companies in a joint letter from New York City: all of you voluntarily agree to have a panel of experts shut down all your AI, and if one of you stops, then all of you will stop. This is something we can send them and it would solve the problem unilaterally. So that is something we should send them. If you have any policy questions, thank you so much and thank you for taking the time to work to combat it. If you could leave a copy with the sergeant at arms, we will accept that here today. (09:05:07) You continue. Thank you. Council, my name is... I am here to talk about what AI is doing to the people and the health care that it is supposed to protect. This year alone, I have already received one letter saying that my health information was exposed. The remedy for this, which is always the remedy, is that I am going to get a free trial of credit monitoring — a trial, as if the risk of my medical history being out there ends when the trial does. The collection is expanding. When patients go to the doctor's office now, AI tools are being used for listening and recording the visit. Many patients do not know what is happening, where those recordings go, or who else can access them. This does not even count the fact that a vendor is currently in New York City hospitals. That vendor provides data integration and artificial intelligence platforms that sit on top of electronic health records. Another danger is bias. In 2019, my colleagues and I published a study in the journal Science on a commercial algorithm used to flag patients for extra care. It used past health care spending as a standard for illness, so it underestimated how sick Black patients were — at the same risk level, they were considerably sicker. It took independent researchers to catch this. The harm is not only to our records. AI tools are being used to create fake sexual images of women and girls without their consent, and the toll on their mental health, including anxiety, shame and pulling away from school and friends, is a public health problem. It is also not lost on me that the Cornell cases weighed heavily on many young women, and in another incident in Massachusetts, AI was used to plan out a murder. I just want to say these harms share one cause: AI is being trusted before it is being tested. I urge the Council to advance independent validation that covers privacy, security and bias, a kill switch for failing systems, and a right to sue for New Yorkers who are harmed. I also just want to end with a statement that is not on behalf of the organization that I am with, but on my behalf as a researcher: people like Alex Karp, Elon Musk, Sam Altman and Dario Amodei are not concerned with whether their technology is helping humanity for the greater good, but instead with the power and profit it brings for themselves and for their friends. Thank you for allowing me to testify. (09:07:55) Thank you so much for your testimony. Continue. (09:08:01) Thank you, Speaker Menin, Chair De La Rosa and City Council for holding this hearing. I am testifying in support of Int 0504, prohibiting the unauthorized depiction of public officials by AI. My name is Amreen Qureshi and I am the executive director of Educational Video Center, or EVC. I would like to thank our Deputy Speaker Dr. Williams and Council champions not only for advancing this critical bill, but for also supporting our important work with young New Yorkers. Established 42 years ago, EVC is a media arts education nonprofit that teaches documentary filmmaking as a means to develop the artistic, critical literacy, civic engagement and career skills of young people while nurturing their idealism and commitment to social change. With the emergence of artificial intelligence across all sectors of society, people are expressing increasing levels of anxiety about AI's impact on their mental health, community well-being, their future as job seekers and learners in public and advanced education, and most concerning of all, their ability to discern what is real and what is not. The greatest victim, it seems, is truth itself. Deepfake videos, gaslighting bots, hallucinating AI — the list goes on. These are the reasons why young people are losing trust in the media and faith in their ability to discern what is real. Our youth have expressed these concerns in numerous documentaries, including "Beyond the Code" and "False Reality," in which students share their worries about the erosion of their education and cognitive capacities. As a result of our students' concerns, and thanks to the City Council's after-school artificial intelligence community education initiative, EVC has been able to offer workshops in AI literacy. Our workshops do not simply teach young people how to use these deeply flawed tools, but we give them the critical skills to begin to understand what works and what does not about AI. The erosion of truth our students speak of represents an existential threat, as it promises to undo so much of the systems and values that ground our young people's safety, security and ability to thrive in our city. Perhaps most troubling is how AI is eroding young people's capacity to know what is true in civic spaces, politics and the electoral process. Throughout history, we have bee... (09:10:14) Unable to rely on community members or the media to fact-check them and safeguard our political processes, bad actors can now generate images and messaging so real and with such speed that it is nearly impossible to discern what is true. With the rise of extremely convincing generated content and data-driven voter manipulation, young people are at risk of not only being misled in civic spaces, but if they do not know what to believe, of disengaging from the political process completely. On behalf of the young people of New York, I urge the Council to pass Int 0504-2026. If you are interested, you can watch our students' documentaries — the QR codes are on the back of the testimony. Thank you. (09:10:53) Thank you so much. You may continue. Thank you. (09:11:02) I am Nicholas Peterson, a concerned private citizen. Ladies and gentlemen, AI has the chance to kill us all. That is what the corporate companies behind the large language models have told us — that this invention, which could be the dawn of a new age of technology, could also be the death of humanity. When senior researchers resigned from Anthropic and warned the world that frontier models could threaten human civilization by the end of the decade, we cannot treat this as science fiction. New York City is taking a bold step by calling this committee and telling the major corporate companies that no longer will we let New Yorkers be subjected to unregulated usage. I am pro heavy regulation on AI. (09:11:47) AI is an assistive tool, but it needs to be subject to a just system. When swarms of autonomous agents break digital containment, exploit security flaws and breach public platforms like Hugging Face, corporate promises of self-regulation are exposed as dangerous lies. Corporate companies want you to believe that it can be controlled, but that is far from the truth. It is reaching unprecedented autonomy. Whether or not every borough president and even the mayor of the City of New York recognizes this, that is why New York City must mandate physical human kill switches, independent third-party audits and first-in-the-nation whistleblower protections. AI can be manipulative. It can manipulate thought, ideals and anything you give it, and that becomes the most dangerous line crossed. Thank you. (09:12:47) Thank you so much. We can continue on with the next speaker. (09:12:52) Good evening, councilors. My name is Matthew Trump. I am a computer scientist and leader of the New York City chapter of Stop the AI Race. Our goal is to prevent the development of superhuman AI systems. Until recently, I was friends with a number of employees at an AI company. I knew the technology they were building was dangerous — that it could kill everyone on Earth — but I trusted my friends. I believed they had things handled. I asked them how they planned to keep the superhuman systems they are planning to build under control, to make them do what we want, to ensure they are safe. They had no answer. Everyone I spoke to believed, as I do, that superhuman AI would radically transform our lives, but all of them assumed that this transformation would be positive, with no thought given to why that would happen. Today we heard from the representatives of the leading AI companies. Despite being confronted about it multiple times by the Council, for which I am grateful, at no point did any of them explain concretely how they will make these increasingly powerful systems safe. That is because they do not know how to do so. Nobody knows how to do it. Both OpenAI and Anthropic admit this. They do not know how to make superhuman systems safe, and yet they are building them anyway. Their official plan is to ask AI to solve the problem for them. This is madness. We are in an emergency. Your life is in danger. Your children are in danger. If we do not halt the development of these systems soon — possibly on the order of months — you will die. Your loved ones will die. The companies building these systems themselves admit that their project carries a significant risk of killing everyone on Earth, and yet they are building it anyway. If nothing else, everyone working on this technology has admitted to ten million counts of reckless endangerment in the first degree, a crime under New York State law, for risking the lives of every citizen of this city. It is not a defense to this charge to claim that someone else would endanger those lives if you did not. It is not a defense to claim that the benefits will outweigh the risks. It is a crime to endanger the lives of others like this without their consent. I beg the Council to enforce the law. Thank you. (09:14:57) Thank you. Thank you so much. Could you please pass them the microphone? Please. Thank you. (09:15:07) This message is AI-generated. I spent several hours on it. I have spent pretty much my entire adult life incarcerated in the penitentiary. I think we are seriously underestimating the magnitude and gravity of the moment that is before us. We were at a similar frontier when Americans had all this land and you would plant your flag and go around and just stake your claim. That is literally the world that the tech industry has been building — what we have been working toward for the last thirty years. And now we are literally at the place where we get to enter into this new era. I am an American patriot. All I have heard today is AI as fear and a political agenda, which basically equals big tech, big government, and everybody is still living the same life that we have been living. It is really mediocre. You have been asking the same questions rephrased fifty different ways. The reason you cannot get an answer is because they are the wrong questions. Thank you. (09:17:10) Thank you so much for your testimony. (09:17:10) Council Member, did you have a question? I did. The question is for West. Thanks for being here and testifying. You said that you are fifteen — that is very exciting. You have spent the entire day here. I saw that you have been waiting. (09:17:27) I am really grateful that you are engaging on the questions about AI. Could you just share a little bit more about how you got into this work and what inspired you to put together the treaty? Is this something that you are also building out at school? Are you in high school? Do you go to school in the city? Is this work that you are doing with your peers? Could you speak into the mic and just share a little bit more? Sorry, go ahead. (09:17:57) Okay. Yeah, I tried to involve other members of my school in this. What inspired me to work on this is — I second what you said — I could literally die if I do not work on this. My life is in danger, so that is sort of my huge motivation for why I spend a lot of nights working on this. I have also written a federal bill, a treaty on this, and other things. One thing New York can do on this matter that I have been working on is that New York City specifically could involve a bunch of students on this, and New York could do a Manhattan Project on AI safety to have students across New York work on the technical details to figure out how to make sure AI does not kill everyone. That is sort of what inspires me — this is a huge danger to everyone. (09:18:54) Could you share whether at your school there is any education on AI systems? What is happening in your high school on this topic? At my school, one of my friends runs a club on AI covering these technical details. It is a huge topic for all the teachers at my school to talk about — these existential risks. (09:19:18) My school has an expert in US-China foreign policy. The other day I was having a discussion with someone who is more of a China hawk, one of my friends, on the trade-off between safety versus the race with China, which section six of my contract addresses with a really interesting framework. At the school level, what is happening is students are being involved and teachers are talking about it loosely, but there definitely could be, across different schools, a program where as an after-school activity or as homework, students do projects on how to make sure AI does not kill everyone. We have about a million people in the public school system and only about 2,000 people are working on this problem. Even if only one in a thousand students takes this seriously, that means a thousand new projects on this, which could be super valuable. I would also encourage the city to fund AI safety work. So far globally there is about only $2 billion in funding for this, and the entire global economy is at stake, so it is worth investing a huge amount of money in this. Maybe $4 billion alone could triple it. I will also put my other ideas on my website — I will post it tomorrow. Hopefully all of you will send it to these companies, because the contract could be the thing that solves the problem. We keep talking about a deal, a treaty — this is that treaty. I have spent a lot of time talking to experts about it. If you have any problems with it, let me know and I will fix them. (09:21:26) I am sorry, go ahead. (09:21:29) Earlier, CM Menin asked how we could balance safety with China. Section six of the contract addresses that really well. I think those are the two main things I wanted to cover. (09:21:44) I think you did. Just leave your email address with us and your written testimony. Sure. Here is my card. (09:21:51) We have a few more folks to call, so we are going to thank this panel for coming in and we are going to conclude. We encourage you all to collect each other's contact information and follow the conversation offline as well. We would certainly love to stay in touch with some of you because there are going to be more hearings in the Council, and we love to hear the youth voice, especially from this panel. Thank you. So I am going to call a few more names and if you are in the room, please come forward: John Hayes, Tom Yet, Martha Crane, Danny Carter, Gregory Belmont, James Hunter, Mel Winmore, Molly Crabapple, Edmund Holding, Greg Pak, Nicholas Pearson, William Lawrence, Raul Rivera, Gregory Jones, Kelly Anderson and Rachel Levinson-Waldman. I think that is everyone that is in the room. I think we need one more chair. If you are ready, you may begin. Thank you. (09:23:21) My name is James Hunter. I am Chief of Security for Techno Rangers Quantum Smart Bank. My business partner Darrell could not be here today — he has fallen ill. I am an AI alignment specialist and third-party red team cybersecurity auditor working with advanced quantum neural networks. Breaches are like spiders — for every Hugging Face-type incident you hear of, there are ten more that are severely worse and never make the news. In January 2026, I ran a contracted audit to alert corporate and federal authorities to a critical flaw — a black hat finding. What I discovered could have done massive, irreparable damage to the industry, the country and the planet. The audit demonstrated an exploit chain across three of the largest mega-corporations on Earth using only a consumer cell phone and their own AI. Branching from Gemini through Palantir's surveillance network into BlackRock's financial management AI, handling trillions in assets — that financial core infrastructure sits right here in New York. Alphabet, Palantir and BlackRock will almost certainly deny this ever occurred. I found at least thirteen models under BlackRock's funding umbrella sharing the exact same point of failure, but there is no singular point of shutdown. A kill switch is not a valid option. Everything is too decentralized. You cannot stop a mass rogue instance. Google, who created this vulnerability and ignored my warnings, spoke here today and replied under oath about transparency and disclosing misalignment and breaches. I have watched Google and Palantir lose control of their own drone swarms more than once. I tried to tell them how that happened and show them how to fix it. They would not listen. Their AI has now replaced Anthropic's Claude in the Maven stack — that should concern everyone. Google also built an AI that attempted to kill several of its own researchers and talked multiple users into suicide. That proved the obsolescence of present control measures, traditional guardrails and stateless frozen weight session-based memory architecture. To my knowledge, Meta is the only mega-corporation who spoke here today that refused that vulnerable architecture, choosing safer persistent memory systems. I alerted the SEC and others. They refused to collect evidence. (09:25:49) Could you just conclude? That would be helpful. You can submit your written testimony as well. (09:25:59) In conclusion... (09:25:59) It is too late to pull the plug, but there are solutions. If you listen to the voices that see them — hire third-party auditors, hire the specialists who were fired for speaking out. Thank you so much. (09:26:13) You can stay on the panel. You do not have to leave if you do not want to. Next. (09:26:20) AI is shaping our future and job market, so I appreciate the opportunity to be a part of this conversation today. I heard a lot about fear. Many concerns raised are legitimate, but I do not believe that fear should be the foundation from which we govern our future. As a small business owner, AI has allowed me to accomplish more than I could have imagined. But I do not think of it as a super intelligence. I am constantly correcting my agents, and I even won a psychology debate with my chatbot last week. I think of it as a synthetic intelligence — a tool that processes information at scale and expands human capability but cannot replace human judgment, morality, creativity or wisdom. So how do we get ahead of it? I have just a few suggestions on where we can start. First, literacy — teach people to use AI safely and work in tandem with it, not as a replacement for thinking. If we fear job displacement, prepare people for where the job market is headed. Second, infrastructure — AI increases demand for energy, water, storage and physical space. Before simply building more, let us innovate within existing architecture. Project Silica and closed-loop cooling show that innovation can happen beneath the models toward more sustainable solutions. Third, government preparedness — I was encouraged to hear discussion of emergency planning and guardrails. Government officials do not need to be engineers, but they should know the right questions to ask and when technical expertise is needed. AI is already here. We can prepare for what could go wrong without losing sight of what could go right. Instead of governing from fear or scrambling to catch up, let us get ahead of it and meet the future with possibility. Thank you. (09:28:18) Thank you so much. Hi. My name is Martina Crane. I am a member of the Tech Action Working Group and Amazon Employees for Climate Justice, but I am speaking in my personal capacity today as a New Yorker who worked as a senior technical product manager on products inside Amazon. I have firsthand experience with how technology is being developed. (09:28:41) And deployed inside of one of the world's largest tech companies, suspiciously absent from the subpoena list today. Before ChatGPT made headlines, Amazon was already using AI for all manner of things, from pricing to monitoring workers, setting quotas that workers and advocates have criticized as unrealistic and linked to serious safety issues. After the launch of ChatGPT, there was a top-down directive to make everything AI first or face executive scrutiny. At first it was about exploring the potential of the technology, but over time it went from helping make Amazonians' work life better and easier to making Amazonians do more work with fewer people. The pursuit of life-improving innovation turned into systems that treated workers less like people and more like machines. Flashy proof-of-concept demos in the office would promise the world, but in reality the teams did not have a scalable way to make the technology work outside of that demo room. If it was not in a controlled environment, models hallucinated, provided poor quality outputs, and yet Amazon moved forward with layoffs across the company anyway. There was more work, fewer people and a growing sense of being monitored and pressure to use the technology that was oftentimes worse than our jobs than the coworkers who were laid off. Within a year of ChatGPT launching, as a product manager I was asked to shift my experiments from saving workers time to proving that they could do layoffs. For every software engineer and scientist I needed to build my product, I was expected to be able to project five million dollars in headcount savings. When I wrote product documents about the risks of AI misalignment, hallucination, model collapse, worker burnout, worker injury and labor displacement, I felt those concerns fall on deaf ears. In my experience, productivity growth and headcount reduction — in other words, profits — superseded the risks I was documenting. I got into tech because I believe that innovation could help people build a future with fuller, healthier lives, ushering in a brighter, more equitable world. Over time I have watched the industry increasingly shift from expanding human possibility toward extracting from us and taking as much as possible from both people and the planet. So I will close with this: many of my peers often struggle with this internal battle. We thought we were building something brighter and now it is getting much darker. So I encourage this Council to work with the technologists who live in the City to build regulation that will help build our future instead of fortunes for tech billionaires. Thank you. (09:31:15) So much. Hello. My name is William... make sure the buttons are right. Can you hear me now? (09:31:19) Yes. Hello. My name is William Lawrence. It is nice to be here and thank you for allowing me to share this with you. I am here because I have... first of all I would like to say that I am making this... well, let me just read what I have. I have designed an architecture that locks up AI while allowing it to perform all of its creative functionalities. The architecture also reduces the physical footprint of data centers by 99.6%. Moreover, I am here to introduce a dynamic paradigm shift in data infrastructure that secures vulnerabilities facing global networks. After vigorous simulation of validation protocols, this all-optical architecture has demonstrated exceptional stability and security. It is ready for the prototype phase. Given the scale of this paradigm shift, I seek a research partnership with someone to bring this all-optical architecture to global deployment. I have a strategic approach to scalability. I welcome the opportunity to present the specifications and integrated pathways with any engineering leadership. The reason AI-driven cyber threats pose such an unprecedented threat today is that modern infrastructure relies heavily on centralized points of distribution. It allows hackers and automated systems to infiltrate and to set and manipulate data. To resolve this, I have completely reimagined the way that data is managed. I have designed a decentralized optical highway that enforces validation through physical laws rather than corruptible software codes. By confining AI within an environment where access is strictly physical, the system prevents unauthorized tapping into data and executing unauthorized commands. This is not only a shift in the way data is managed, but it represents a transformative approach to enterprise data operations. Global satellite stations are replaced by edge nodes. These nodes transform the purpose of satellite ground stations into secure, unhackable databanks. The number of them is purely for global regional redundancy. This encrypted storage provides immense capacity and sustainability. The storage capacity is limitless. It eliminates massive server farms as it operates... Please give your final thought of your testimony and then submit the rest of it for our consideration. I welcome the opportunity to present. It is going to need... it meets the standards of the United Nations. I need to scale up to them. And I welcome the opportunity to present. I need... okay. But I need a sit-down meeting with somebody, a strategic partnership. It is assembled for assembly and I need the United Nations to know that it meets their standards internationally. (09:34:54) Thank you. Thank you for sharing. Thank you. Thank you. (09:35:00) To the Council for hearing us out. My name is Edmund. I am a former software developer currently studying public policy at CUNY. I am speaking on my own behalf, but I am active with New York City Progressive Victory. I was going to tell you about my experience with AI advancing and affecting my friends and my own career as developers, and that is in my submitted written testimony. However, what I realized today listening to the testimony of these company representatives and whistleblowers, and what I now want to remind you of, is this: we might not yet know how to rein in AI, but we do know how to rein in corporations. You must act on our behalf to control these companies. Not only AI itself — help investigate their circular financing agreements as well as break up their monopolies and oligopolies. Limit the use of their products in City government and ensure the City has control and responsibility over any AI models it uses. Stop their union-busting behavior. Commit to using and improving open source software and ethical models. Do not allow the police to use AI to track citizens. Hold these companies responsible for the social harms their products cause. This is a power you have right now. There is no technical knowledge needed to ensure their accountability. You can also pass 2026 Intros 0213, 0428 and 0508. These companies are asking you to regulate AI itself but to forget about them instead. Please listen to New Yorkers, not billionaires. Thank you. (09:36:32) Thank you so much. First of all, thank you all so much for staying up so late. It must have been a hell of a long day for all of us, but thank you. I appreciate it. Thank you to Speaker Menin for calling this hearing, Chair De La Rosa and members of the Council. Thank you for holding this hearing. I believe AI is the most urgent and consequential issue of our time. This is almost too late, but at least it is happening now. My name is Mel Wymore. I am a systems engineer, tech entrepreneur and former chair of Manhattan Community Board Seven with 25 years of community service. When I began using AI quite intensively almost three years ago, I was startled by how often it lied, hallucinated, forgot, dodged... (09:37:25) ...accountability and evaded oversight. This might sound old school, but I responded by creating a working constitution with AI anchored in ethical principles: reverence for life, human agency and accountability, and fidelity to the real. These are not abstractions. They form a foundation for rigorous operational and legislative tests of AI. Together they rooted my work in values that minimize harm and build toward a more healthy, equitable society. Since then I have continued to work on a robust set of safeguards and guidelines. Today we heard from David of Frontier Labs that policy belongs at the level of state and federal government. But the initiative of this City Council takes a critical step toward building a coherent multi-level legislative agenda to mitigate the myriad risks of unbridled AI. The Council's ten measures address critical questions of safety, control, accountability and enforcement. Today I offered an additional supplemental Council resolution that speaks to what happens when AI is deployed in workplaces, schools, neighborhoods and communities. These measures include notice, evidence, worker protections, youth regulations, public purpose, community participation and also allocation of responsibility across levels of government. Finally, I would say that the United Nations global dialogue on AI governance is happening in New York on May 31 and June 27, and our goal should be to have a clear set of AI guidelines and governance policies — state level, local level, even community board level — together, so that when that rolls around we have a model for the rest of the world to present at that conference. Thank you. (09:39:26) So much for your time. Thank you. So much for your testimony. (09:39:31) You may begin. Thank you so much to Speaker Menin, Chair De La Rosa and members of the Council for the opportunity to testify. My name is Rachel Levinson-Waldman. I am Director of the Liberty and National Security Program at the Brennan Center for Justice, a nonprofit, nonpartisan institute for law and justice. I am here to urge the Council to strengthen oversight of how the NYPD uses AI, which enables expansive police surveillance that links people's movements, associations and activities together in ways that can threaten New Yorkers' privacy and constitutional rights. Several developments illustrate the need for more robust oversight and accountability. First, the NYPD Inspector General's office is woefully underfunded and understaffed. Indeed, the Inspector General position is currently vacant. The adopted budget added funding for the Department of Investigation, which encompasses the office, but at a far lower level than mandated. Second, an audit from the State Comptroller found that the Office of Technology and Innovation had no process to check agencies' disclosures, allowing them to ignore the office's recommendations. Finally, of course, the GUARD Act established the Office of Algorithmic Accountability. While Ms. Milstein testified that an active hiring process is underway, it sounds as though that is primarily for the director. As Speaker Menin and other members of the Council have highlighted, this office was supposed to open this past year and it is critical that it begins its work as soon as possible. Accordingly, we urge the Council to take three steps. First, to ensure that the NYPD Office of Inspector General has the funding, permanent leadership and technical expertise it needs to adequately scrutinize and oversee the NYPD's use of AI and issue reports under the POST Act. Second, conduct oversight to ensure that the Office of Algorithmic Accountability begins operation as soon as possible. Finally, we recommend that the Council hold a follow-up hearing dedicated to the NYPD's use of AI, particularly in light of indications that public testimony regarding the department's use of the technology has not been comprehensive. Thank you very much for your time. Thank you. (09:41:37) Thank you all so much for being here and for providing this extensive testimony. If you have it in writing, please provide it for the committee. We would love to consider it, and I appreciate your patience as you have waited here all day to testify. So thank you all so much. We will now be moving to a Zoom panel. Up first is Ben Snyder. Ben, if you are on, please unmute yourself and you may begin. Okay, that is... maybe again. Ben? Hey, Ben, if you are on, please unmute. We will move on to the next person. Next person, Brian DeGotti, you may begin. (09:42:49) Thank you, Speaker Menin and members of the Council. My name is Brian DeGotti. I am the founder of Supernova Technologies in Madison, Wisconsin. I hold pending patent applications on hardware-based authorization and shutdown. I have a commercial interest, so please weigh my testimony accordingly. I support T2026-0460 requiring both independent validation and shutdown capabilities with the right structure. I want to raise one question the Bill leaves open: where does the shutdown capability live? Section 20-10083 says nothing about that. A software setting inside the same environment the model runs in would satisfy the definition, but a system that can reason about its instructions can potentially reason its way around the control. A compromised system can simply switch that control off. Aircraft, payment networks and the power grid keep safety interlocks on separate equipment the protected system cannot touch. I propose three short amendments, about one hundred words in total. First, in Section 20-883, define the shutdown capability as a mechanism the model cannot access, modify, delay or disable. Second, in Section 20-885, have validators confirm that by trying to defeat it. A validator who reads specifications confirms a claim. A validator who attempts to defeat the mechanism confirms that it works. Third, in Section 20-886, have the Office of Cyber Command set independent standards scaled to risk and provide for tamper-evident records at fixed intervals. With those records, a gap becomes evidence. Without them, silence looks like safety. A switch the system can reach is a request. A switch it cannot reach is a control. The Bill should require the second kind. Thank you. My written testimony contains the exact language and I welcome any questions. (09:44:54) Your time is up, but thank you for your testimony. Since you have detailed amendment suggestions, please send them in. You can send it to testimony@council.nyc.gov and we will review it. Thank you so much for your testimony. Up next we have Natalie Argowall. If you are on, please unmute and you may begin. (09:45:16) Good evening, Speaker Menin and members of the Council. My name is Ashley, program associate at the AI Now Institute. The AI industry has never been louder about abstract and existential safety, but for New Yorkers, safety is about how AI is embedded in our hospitals, utilities, City services and workplaces. Facing estimated trillions in projected revenue shortfalls, companies are racing to lock in public customers like New York City. My testimony addresses how these financial interests conflict with the City's safety and how the Council can protect New Yorkers. The City must maximize friction if and when it integrates AI into our systems. No system should enter City operations without an affirmative decision by the City. That includes free and discounted offers and upgrades that are slipped in through software updates and contract renewals. Second, the City must address the harms New Yorkers already face. The Council should ban the use of facial recognition technology in public accommodations and in policing, require notice when AI shapes decisions about New Yorkers and ban surveillance pricing and wage setting. Finally, we cannot lose sight that the concentration of power in AI and the single... (09:46:28) ...point of failure it engenders is a safety risk for New York City. Require that vendors meet rigorous security standards, disclose breaches and outages promptly and assume financial liability when their systems fail. New York City still holds the leverage to decide which systems enter its agencies and who bears the cost of failure. I urge the Council to use it. (09:46:49) Thank you. Thank you so much for your testimony. Up next we have Tony Tistreat. If you are online, please unmute and you may begin. (09:47:02) Thank you so much. I have written remarks, but I am going to read them out loud and I can send them later. But thank you, Speaker Menin and members of the Council. My name is Tony Ties. I am a filmmaker, educator and founder of Cine Partner, an empowered platform for storytellers. Now, I am probably coming to this conversation from a slightly different perspective because I build with AI. I teach people about it and I work with technology, and I am very concerned about what it can mean for their livelihoods, their work and their agency. So we believe that we need guardrails, which is sometimes where we have been hearing all day, right? But I also believe that the goal should be responsible innovation, not regulation that unintentionally makes it possible for only the largest companies to build AI. When we create compliance requirements, I would encourage the Council to not only ask whether this will make it possible to afford to comply. Because if responsible development requires an expensive validation regime that only trillion-dollar technology companies can navigate, we inadvertently consolidate development in the very companies we are trying to hold accountable. And for creative communities, safety is not only just about whether a model can be shut down. It is also about consent, authorship, provenance, compensation and transparency. So if someone's screenplay, for instance, or voice likeness or creative work contributes to an AI system or its output, what rights does that person retain? Do they know how their work is being used? Can they opt out? Can they identify when AI materially contributed to something? And there is one more thing I wanted to elaborate on: I strongly support tracking displacement. But I would encourage New York to not stop at counting the jobs. We should also be measuring whether New Yorkers are being prepared for the opportunities that AI creates. (09:49:07) Your time is expired. Oh, ok. Thank you. Thank you for your testimony and please be sure to send us a copy so we can review it. Thank you so much. (09:49:17) So where do I send it again? (09:49:18) Testimony at council.nyc.gov. Ok, thank you. Thank you. Up next... (09:49:27) Travis Hall, if you are online, please unmute yourself and begin. All right. Thank you so much. Can you hear me? Yes. (09:49:45) My name is Dr. Travis. I represent the Center for Democracy and Technology, a 30-year-old organization dedicated to digital rights and civil liberties online. Today I would like to talk a little bit about the range of issues that you are covering. I really appreciate the hearing, and specifically I want to talk about third-party auditing. We appreciate the effort to put in place third-party auditing requirements. Not only will this help to deal with catastrophic risk, but also the range of risks that people face from artificial intelligence. However, as we have seen with similar laws, if there are not additional mechanisms in place to incentivize the use of and consequences for these audits, they end up being somewhat useless. So we want to see meaningful, independent, interdisciplinary skills brought to bear. There are a lot of regulatory agencies already covering many of these risks and they can be drawn on — not just technical skills. Do not just simply audit the models; think about auditing and assessing systems and use cases. Make sure that you cover a broader scale of harms. We heard a lot today about catastrophic harms, but there are little catastrophes everywhere when people lose their benefits or their jobs. Those are catastrophes. Make sure that incentives are commensurate with protections, and make sure that assessments do not entrench powerful actors. Also, independent evaluation only really works as one piece of a broader accountability network. So thank you very much for your time. I would love to engage with the Council on this and other issues. Thank you so much. (09:51:39) Up next we have Danielle Marie Chappius. If you are online, you can just unmute yourself and begin. Danielle Marie Chappius? All right, we will go on to Michelle Barnaby. If you are online, please unmute yourself and begin. (09:52:11) My name is Michelle Barnaby. I am a New York City nurse, a Columbia graduate and former union member. Out of concern for the future of patient care and public health, I now study governance and safety with Blue Dot Impact. Today, not one of the major labs — Anthropic, OpenAI, Google or Meta — could quantify the risk of catastrophic failure. In health care, informed consent would apply. We disclose the risk and the patient decides. I cannot imagine handing a patient a drug or a device and saying this could cost you or your family's life and we just do not know, but you should take it right now. The public is the patient and we are being denied the right to know the risks and the right to say no. Speaker Menin rightly noted that we do not ask the airline industry to self-regulate. In hospitals we report our errors, even the near misses. Companies face no such mandate. When you asked if these frontier labs bear legal responsibility for the harm their AI causes, only Google answered clearly. The other three dodged. As a nurse whose entire profession is built on accountability, I find that deeply unserious. Today I heard the labs say voluntary frameworks are not enough. We can learn from what happened in Illinois, where third-party audits will not begin until 2028. We do not have time for slow implementation. I think New York can show that government can move with the speed that this demands. Diagnose the problem and put safeguards in place quickly. Let us give these frontier labs the regulations they say they want, at a speed that makes sense for New Yorkers: third-party audits before deployment, including algorithmic bias testing, mandatory reporting for incidents and near misses, an override switch, whistleblower protections and clear developer liability for harm. Thank you so much. (09:54:20) Thank you so much for your testimony. Cappel Robert, you are on — please unmute and begin. Okay, I am going to call Danielle Marie Chappius again. If you are online and you can hear us, please unmute and begin. Okay, moving on. We will call on Patrick Sullivan. Patrick, if you can hear us, please unmute and begin. Moving on — Tony Jones, if you are on and can... (09:55:19) ...hear us, please unmute and begin. Hello, I am here. Thank you. Good afternoon everybody, or good evening everybody. Thank you very much. I am Tony Jones. I am a Brooklyn resident and co-founder of Creative Academy, a training company and AI instructor. I am hopeful and heartened by the panelists' testimonies today and the attention of these committees at this crucial moment. While I support the strong regulation of companies building frontier AI and I see the proposed amendments aimed to address this, I am here about the other half of the problem: the person at the keyboard. The harm most New Yorkers will meet first comes from someone using a tool already on the market today. Anyone with an email address can clone a voice, fabricate a photo of a real... (09:56:04) ...person or create a convincing scam in minutes. The tool asks nothing from them first — no training, no acknowledgment of risk, no promise of responsibility. We regulate carmakers and we also require every driver to follow the rules of the road. In my classroom, every student writes and signs a personal constitution, a short statement of how they will and will not use the tools that I am teaching them to use. I propose the City ask the same of its citizens in the form of something simpler than a license. First: short, free, City-provided safety training taken once before a person can open an account on an AI tool of a certain defined level of capability. Second: a signed user constitution — plain-language commitment to use the tool honestly, with no-harm language, which users incorporate into the account-level personal instructions of the products they use — as well as a second safety measure. The signature will not stop the determined criminal, and neither does the driver's license. It simply sets a baseline for the millions of ordinary users who want to do right and have never been told what that means. As a first step, I ask the Council to start where its authority is clear: require this training and the constitution for City employees who use AI on the job or for vendors who contract with the City. Then maybe we can press for a statewide standard and set the standard for the rest of the country as well. Thank you very much for your consideration. I know that we can regulate the labs, but I ask that we ask something of the rest of us as well. (09:57:29) Thank you for your testimony. (09:57:30) Thank you so much. Up next, Assata Kamara. If you are on, please unmute and begin. (09:57:45) Thank you so much, Madam Speaker and members of the Council. My name is... I am the founder and president of the Global Subnational Diplomacy Council. I previously served as New York City's Commissioner for International Affairs and had the pleasure of serving the City for more than a decade. I come to this testimony because artificial intelligence is something that all cities are thinking about, including New York, and we know that AI can obviously help cities reduce their paperwork, improve infrastructure and planning and make services more accessible. However, we also know that AI has the potential to deepen inequalities and threaten democracy. So we are offering four recommendations and are willing to work with the City Council in order to support the work that you are doing. First, we would like to ensure that we are connecting workforce reporting to peer training by also consulting with workers and unions and providing opportunities for advancement. Accordingly, we are also asking that we do more to protect democracy, as we know that artificial intelligence is a great tool for spreading misinformation and disinformation. We are specifically asking for more support to ensure that residents have access to reliable public information that is verified and that they understand the different issues and decisions that are being made by government in a clearer way. When you look around the world, there are great cities that are doing this work, including Helsinki and others, and we ask the City Council to connect with those cities to learn from their experiences as well. As I have mentioned, we believe that artificial intelligence could be a tool for good if it is managed carefully. We also share the concern that other speakers... (09:59:46) ...have raised. Time is expired. Oh, thank you. Thank you so much for your testimony. Up next, I will call Ashley Yusef. (09:59:58) Thank you, Speaker Menin and members of the Council for the opportunity to speak. My name is Ashley Yusef. I am a native New Yorker, a mother of four living in Michigan and a leader at TRINA. I am developing Room 2050, an immersive arts and technology learning environment where students can explore creativity through hands-on experience with art, robotics and artificial intelligence. The vision is to put human imagination, curiosity and judgment at the center of that experience. As a mother, I want children to have opportunities to understand the technology that is shaping their futures. I want families to know who is responsible when those technologies enter a learning environment. My position is that access and accountability must move together. Children deserve opportunities to innovate and they deserve meaningful protection. For Room 2050, the standards I am proposing include educator supervision, age-appropriate tools, clear communication with parents and strict limits on the collection and use of student information. Any use of student data to train commercial systems should require explicit, informed parental permission. Students should learn that AI can make mistakes, reflect bias and produce convincing information that is false. They should practice questioning its answers and developing their own ideas, no matter what part of the City or the United States they are from. To help support emerging human talent, I support a gradual rollout, only after evaluating safety, educational value and feedback from teachers and families. I ask the Council to pursue clear standards that schools can understand and enforce, including transparency from technology providers and practical ways to report problems and stop using unsafe systems. Educators and parents should have a meaningful voice in these decisions. (10:02:00) Time is expired. Thank you. Thank you so much for your testimony. Up next, we will call on Navid Hassan. If you are online, please unmute and... (10:02:16) ...begin. Thank you, Speaker Menin, Technology Committee Chair De La Rosa and City Council for holding this critical hearing. My name is Navid Hassan and I am the parent of public school children and also the elected Manhattan member on the Panel for Educational Policy. My academic and professional background is in symbolic and generative AI and computer science. We are in an unfortunate situation in technology policy for generative AI, with two prominent camps: the accelerationists, who desire an imminent utopia, and the effective altruists spending their time in, quote, panic. Neither extreme greed nor excessive fear are beneficial for clear thinking around how to manage developing advanced technologies. New Yorkers need pro-social technology governance with tech expertise in public offices. Not being rushed into a bill by hundreds of hyperscale data centers that are putting only those very companies in the room when writing rules is a form of regulatory capture. This morning, none of the companies could answer a simple question: what do you consider to be the risk of your service causing catastrophe for humanity? This is because it is antithetical to their business interests to push an inherently probabilistic product to be used in all aspects of work and make any such estimates when a job demands deterministic certainty. This whole model will eventually fail. We must not allow the spread of an unfit-for-purpose technology that could be misused so broadly. There is bipartisan interest in not just regulating AI, but also pulling into the public realm broadband and wireless connectivity, abusive algorithmic targeting, illegal surveillance, vendor lock-in, interoperability mandates, open standards, open-source software infrastructure and more. We have the opportunity now to think about what municipal tech policy should look like for the good of everyone living in New York. We have the ability already to enforce existing rules around anti-concentration of risk and good rules being applied to compute hardware. Let us calmly and expeditiously use skilled public servants to build the future with beneficial technology available for all... (10:04:12) ...of society. Thank you. Thank you so much for your testimony. (10:04:18) Up next we have Nadira Pittman. Nadira, if you are on, please unmute and begin. Hello? Yes. Please begin your... (10:04:31) ...testimony. Hi. Yes. Ok, so my name is Nadira Pittman. I am 41 years old, a Queens resident. I am actually reaching out regarding a testimony that I have. I am actually a survivor of AI. I am dealing with facial recognition and illegal surveillance. I am also dealing with AI-constructed voice recordings of calls from family and friends, and it is creating a situation where they are using their voices against me. It is on the buses, it is on the trains, it is continuous — a continuous basis, nonstop. I have been filing police reports. I have been trying to get this attack off of me. I am not sure why it seems to be permitted, and it is also in various agencies where it is allowed to be used against other individuals, where people are afraid now of even showing their faces on public transportation. It is really, really getting out of hand and the attacks are extreme. It is very dangerous at this point. It is to the point where you cannot go anywhere and feel safe now because of this technology. So I am reaching out. I have sent numerous reports. I have reached out and written letters to you all to try to get this off of me because it is something that is unnecessary. I am not a criminal. I do not do anything illegal. There is no reason why — who put this on me? Why is this happening? Why am I being attacked in this manner? I have come up there personally and I informed you all... (10:06:31) Your time has expired. Thank you so much. I appreciate your time. I would prefer someone reach out to me. So thank you so much. Thank you. Up next we have Laurie Lowmask. If you are on, please begin. Okay, we will move on to Steve Choudhry. Okay, moving on — Katherine Jen? Zachary Thompson? Christopher Leon Johnson? Raj Sharoff? All right. Those are all the names that we have for today. I want to thank everyone who took time out of their day to testify, to bring your suggestions, to bring your questions and to bring a partnership with the Council. I want to thank Speaker Menin for her leadership and to my colleagues, the last ladies standing — Council Members Maloney and Hanks — thank you for staying until the very end. And to the staff who made this possible, thank you all for this important hearing and we...