Overview

This self-driven data analysis project explored patterns of police misconduct within the NYPD, leveraging newly available Civilian Complaint Review Board (CCRB) records following the repeal of Law 50-a, which had previously shielded officer disciplinary records from public scrutiny. The dataset, obtained by ProPublica, includes allegations against NYPD officers, demographic details of both officers and complainants, and investigation outcomes.

My analysis focused on understanding trends in police misconduct, racial disparities, and disciplinary actions by:

• Examining the dataset schema, which consists of 26 attributes per officer, including misconduct type, officer demographics, and complaint resolution.

• Preprocessing and cleaning the dataset, addressing missing values and inconsistencies to ensure reliable insights.

• Conducting data exploration to uncover trends in misconduct frequency, substantiation rates, and disciplinary inconsistencies.

• Investigating racial disparities in how complaints were filed, processed, and resolved, examining potential systemic biases.

By structuring this project as a data-driven investigation, I aimed to provide a clearer picture of accountability and transparency in NYPD disciplinary processes, using data science as a tool for public interest research.

Motivation Statement

This project was driven by my interest in data ethics, justice, and the role of transparency in accountability. The repeal of Law 50-a marked a pivotal moment for policing oversight in New York, enabling researchers, journalists, and data scientists to scrutinize misconduct cases that were previously inaccessible. I saw this as an opportunity to apply data science in a meaningful way, analyzing patterns of misconduct and systemic disparities that affect public trust in law enforcement.

Beyond the technical challenge of working with real-world law enforcement data, this project allowed me to engage with important social issues—exploring how data can highlight systemic trends and inform discussions on police accountability.

This aligns with my broader commitment to leveraging data science and AI for social impact, ensuring that technology serves as a tool for transparency and reform rather than reinforcing existing biases.

Reflections and Learnings

Working with real-world law enforcement data presented unique challenges and insights, strengthening my ability to apply data science in investigative contexts. Key takeaways from this project include:

1. Data Science as a Tool for Public Interest Research – This project reinforced the idea that data can be a powerful force for accountability, allowing for deeper scrutiny of policing practices and systemic biases.

2. Handling Imperfect Real-World Data – Unlike structured datasets designed for controlled analysis, law enforcement records contain missing values, inconsistencies, and ambiguous categorizations. Cleaning and structuring this data required strategic preprocessing techniques.

3. Translating Data into Meaningful Insights – Extracting trends from raw data was only part of the challenge; framing those insights in a way that contributes to public discourse on justice and transparency was equally important.

4. Ethical Considerations in Law Enforcement Analytics – Working with sensitive data reinforced the importance of responsible data analysis, ensuring that findings were contextualized and that privacy considerations were respected.

This project strengthened my passion for data science and reinforced my commitment to leveraging data for meaningful societal impact.