New York’s courts decide the overwhelming majority of cases that shape residents’ daily lives—from criminal matters to evictions to family separation. Yet the judiciary remains the least transparent branch of government. In this conversation, Searchlight speaks with Oded Oren, founder of Scrutinize, a nonprofit using public data to illuminate how judges make decisions and how those decisions affect New Yorkers. Oren explains Scrutinize’s mission, its growing suite of judicial metrics, and its newest project—Decoded, a first‑of‑its‑kind effort to map the impact of every New York Court of Appeals decision on core rights and everyday life. What follows is an edited transcript of our discussion.
Searchlight: Oded, let’s begin with the basics. What is Scrutinize, and what problem was it designed to solve?
Oded Oren: Scrutinize is a nonpartisan nonprofit focused on judicial transparency and accountability in state courts, especially New York. We collect public data—either already online or obtained through FOIL requests—and analyze it at the level of individual judges. Our goal is to make judicial performance visible to the public.
Searchlight: When did Scrutinize launch?
Oren: We founded Scrutinize in 2023. I had been a public defender in the Bronx and left to start it. Our first report, released in May 2023, analyzed bail determinations. Since then, we’ve expanded our metrics and, with Decoded, moved from trial courts to the state’s highest court.
Searchlight: For readers who may not know: New York’s highest court is the Court of Appeals, not the Supreme Court. Why focus on state courts?
Oren: Because state judges don’t have lifetime tenure like their counterparts on the federal bench. State judges must seek new terms—through elections, reappointments, or other processes. That means the system expects oversight by the public over the judiciary. But such oversight is impossible without accurate information and data of their decisions and impacts. We aim to close the information gap by providing the data that feeds back into a public, evidence-driven understanding of how judges are using their judicial power. Our data informs the decision of whether to reappoint or reelect them.
Searchlight: From your trial‑court analyses, what patterns emerged?
Oren: We’ve released nine analyses so far. Each quantifies a specific judicial decision—bail, reversals, sentencing—and maps how judges compare within their county or across the state. You see expected variation, but also clear outliers. Those outliers have prompted activism: New Yorkers are using our data to push decision‑makers to reconsider reappointments or promotions.
Searchlight: So the advocacy is aimed at appointment processes rather than changing how judges are elected?
Oren: That is the advocacy that we’ve seen so far. Activists identify judges with extreme results from our data and urge the mayor, governor, or court officials not to reappoint them. They also call for more transparent reappointment processes that include public input.
Searchlight: There’s an obvious tension between accountability and the risk of politicizing the judiciary. How do you strike that balance?
Oren: It’s a complicated question. People will have different views about where to draw the line between criticizing judges, influencing who serves on the bench and protecting judicial independence. Scrutinize’s focus is on providing reliable information. We can examine the actions of mayors, governors and legislators, evaluate their records and use that information when deciding whether to support them. But historically that kind of systematic scrutiny has been much less accessible when it comes to the judiciary.
We’re trying to fill that gap. We don’t take positions on individual judges or tell people how to use the data. Our job is to make rigorous, objective information available so the public and those responsible for judicial appointments can make better-informed decisions.
If we’ve decided, as a state, that judges should not serve indefinitely without accountability, people need information about their performance. Otherwise, decisions about their reappointments or reelections can depend too heavily on anecdotal evidence, political connections or backroom dealings.
Searchlight: How does the platform work? Can people look up individual judges, or do you also publish broader analyses?
Oren: Both. Whenever we develop a new data source and turn it into a metric for evaluating judges, we make the data available on our website. Users can look up a judge, see how that judge performed on the measure and compare the results with those of other judges.
We also publish detailed reports, often around 40 pages, explaining the research behind each metric. We describe why the particular judicial decision matters, why the public should care about it, how we collected and cleaned the data, and how we conducted the analysis. We hold ourselves to the same transparency standards we expect from the judiciary. People should be able to examine not only the results but also the methodology and legal research underpinning them.
Searchlight: Can your metrics identify whether a judge is “tough on crime” or “pro‑defendant”?
Oren: In theory, yes—but responsibly. Some metrics relate to those ideas, but a holistic assessment requires many data sources. We aim to provide that full picture over time, but we don’t label judges. We keep the data neutral.
Searchlight: Who should be using Scrutinize?
Oren: Ultimately, everyone. State courts handle 99% of criminal cases, all eviction cases, all family separation cases—issues that profoundly affect people’s lives. But there are key bottlenecks in the current system that provide some people with even more power over who gets to serve another term on the bench:
- The Mayor’s Advisory Committee on the Judiciary, which recommends judges for appointment or reappointment.
- Judicial delegates in Supreme Court elections, who effectively decide which candidates appear on the ballot.
We want those decision‑makers to use data rather than anecdotes or political ties.
Searchlight: Let’s talk about Decoded. What does it add?
Oren: With Decoded, we’ve created new data about the state’s highest court that was never publicly available. The Court of Appeals doesn’t write many decisions, but each is long and dense, and it’s hard to understand what it does to the law and how it will affect life in the state. So we built a system that takes each such decision and makes it accessible: what the court ruled, what areas of everyday life it affected, and how each judge voted to affect those areas.
Searchlight: What time period does Decoded cover?
Oren: We analyzed all decisions issued since Judge Jenny Rivera joined the court in 2013—she’s currently the longest‑serving judge—through August or September 2026. We’ll update monthly so users can track trends over time.
Searchlight: Does that approach match how you handle trial‑court data?
Oren: Mostly. For trial judges, we usually show data only for current judges, since analysis is time‑intensive and expensive. But we sometimes include former judges to show systemic issues—like constitutional reversals—when historical context matters.
Searchlight: Oded, thank you. Transparency in the judiciary is central to Citizens Union’s mission, and Scrutinize’s work is a major contribution.
Oren: Thank you. I appreciate the opportunity to share this, and I’m excited for people to use the data.