Search This Blog

Showing posts with label research. Show all posts
Showing posts with label research. Show all posts

Wednesday, September 20, 2023

The Changing Politics of Race

Many posts have dealt with racial issues.

Joseph Simonson at The Washington Free Beacon:

Ibram X. Kendi’s Center for Antiracist Research made headlines this month when it announced it would axe a third of its workforce. But those layoffs may not have much of an impact, considering the center has hardly produced any original research at all.

The Boston University-based center has produced just two original research papers since its founding in June 2020, according to a Washington Free Beacon review. Output from the center’s scholars largely consists of op-eds or commentary posted on the center’s website. The group’s plans to "maintain the nation’s largest online database of racial inequity data in the United States" quickly fizzled out, and the database has been dormant since 2021.

The Center for Antiracist Research is the latest left-wing group to fall on hard times. George Soros’s Open Society Foundations, which gave $140,000 to Kendi’s center, cut 40 percent of its staff in June. The Black Lives Matter Global Network Foundation’s revenues fell 88 percent from 2021 to 2022, as support for the movement plummeted to an all-time low.

It is unclear how much money remains in the Center for Antiracist Research’s coffers. Boston University did not respond to a request for comment.

Liam Knox at Inside HIgher Ed:

The Supreme Court’s June decision striking down race-conscious admissions may have been the most significant higher ed case in years, providing a concrete answer to questions that have spurred dozens of court cases since the 1990s. But it hardly put an end to the legal fight over affirmative action.

In fact, the outcome has unleashed a stream of new challenges to colleges’ race-conscious policies and revived cases that had been dismissed or lost before the ruling was handed down.

Just yesterday, Students for Fair Admissions, the group that spearheaded the Supreme Court cases against Harvard University and the University of North Carolina, filed a lawsuit challenging the race-conscious admissions policies of the U.S. Military Academy at West Point. SFFA began building the case after the Supreme Court left open the possibility that military colleges could be exempt from the affirmative action ruling due to their “potentially distinct interests” in enrolling racially diverse student bodies.

Wednesday, August 30, 2023

Wednesday, May 31, 2023

AI-Generated Research and Hallucinations

Artificial intelligence is an increasingly important topic in politics, policy, and law.

Benjamin Weiser at NYT:
The lawsuit began like so many others: A man named Roberto Mata sued the airline Avianca, saying he was injured when a metal serving cart struck his knee during a flight to Kennedy International Airport in New York.

When Avianca asked a Manhattan federal judge  to toss out the case, Mr. Mata’s lawyers vehemently objected, submitting a 10-page brief that cited more than half a dozen relevant court decisions. There was Martinez v. Delta Air Lines, Zicherman v. Korean Air Lines and, of course, Varghese v. China Southern Airlines, with its learned discussion of federal law and “the tolling effect of the automatic stay on a statute of limitations.”

There was just one hitch: No one — not the airline’s lawyers, not even the judge himself — could find the decisions or the quotations cited and summarized in the brief.

That was because ChatGPT had invented everything.

The lawyer who created the brief, Steven A. Schwartz of the firm Levidow, Levidow & Oberman, threw himself on the mercy of the court on Thursday, saying in an affidavit that he had used the artificial intelligence program to do his legal research — “a source that has revealed itself to be unreliable.”

This case was not unique. Gerrit De Vynck explains at WP:

Recently, researchers asked two versions of OpenAI’s ChatGPT artificial intelligence chatbot where Massachusetts Institute of Technology professor Tomás Lozano-Pérez was born.

One bot said Spain and the other said Cuba. Once the system told the bots to debate the answers, the one that said Spain quickly apologized and agreed with the one with the correct answer, Cuba.

The finding, in a paper released by a team of MIT researchers last week, is the latest potential breakthrough in helping chatbots to arrive at the correct answer. The researchers proposed using different chatbots to produce multiple answers to the same question and then letting them debate each other until one answer won out. The researchers found using this “society of minds” method made them more factual.

“Language models are trained to predict the next word,” said Yilun Du, a researcher at MIT who was previously a research fellow at OpenAI, and one of the paper’s authors. “They are not trained to tell people they don’t know what they’re doing.” The result is bots that act like precocious people-pleasers, making up answers instead of admitting they simply don’t know.
The researchers’ creative approach is just the latest attempt to solve for one of the most pressing concerns in the exploding field of AI. Despite the incredible leaps in capabilities that “generative” chatbots like OpenAI’s ChatGPT, Microsoft’s Bing and Google’s Bard have demonstrated in the last six months, they still have a major fatal flaw: they make stuff up all the time.

Figuring out how to prevent or fix what the field is calling “hallucinations” has become an obsession among many tech workers, researchers and AI skeptics alike. The issue is mentioned in dozens of academic papers posted to the online database Arxiv and Big Tech CEOs like Google’s Sundar Pichai have addressed it repeatedly. As the tech gets pushed out to millions of people and integrated into critical fields including medicine and law, understanding hallucinations and finding ways to mitigate them has become even more crucial.

Sunday, January 1, 2023

Interest Group Money and Cannibis Research

When UCLA started its cannabis research initiative five years ago, the university hailed the undertaking as one of the first academic programs in the world dedicated to studying the health effects of pot.

Legalization was quickly taking hold around the country, and the cannabis industry was attempting to transform the plant’s image from an illicit substance that gets you high to a health and wellness product.

The Times asked UCLA officials whether the university accepted donations from the industry to support the program. They said no.

However, documents obtained by the newspaper, eventually released by UCLA under the California Public Records Act, show that cannabis companies and investors provided at least some of the early financial support, writing checks for tens of thousands of dollars in donations and assisting with fundraising events.

The industry support underscores potential conflicts of interest as pot goes mainstream and researchers try to assess the health and other effects of cannabis. A marijuana investor and foundations with ties to the newly legal cannabis industry have donated millions of dollars to university research programs studying claims of the plant’s medical virtues, raising questions about how independent the scientific research can be.

Harvard, the Massachusetts Institute of Technology and UC San Diego are among the schools that have accepted multimillion-dollar gifts in recent years....

Private industry funding of biomedical research has become increasingly common over the decades, to the point where it is now the largest source of funding for research. Past studies have shown industry-funded research has a greater tendency to produce results favorable to the industry, according to Joanna Cohen, professor of disease prevention at Johns Hopkins Bloomberg School of Public Health.

“The research is strong enough that we know the source of the funds is problematic,” Cohen said. “There’s no reason to think cannabis will be any different.”

In 2003, a study conducted at the Yale School of Medicine found that industry-funded studies were 3.6 times more likely to produce outcomes favorable to their sponsors.

Sunday, October 10, 2021

Research 2021

 Watch this clip from The Wire.  It is the best description of research, ever.

The Internet Archive -- if there is a broken link to what you need, this site might help you find it.  (A newsworthy example, the International Chiropractors Association scrubbed this anti-vaccine diatribe from its website: http://www.chiropractic.org/?smd_process_download=1&download_id=3683

Great stuff at Honnold Library -- which students usually overlook! (password required)
  • Nexis Uni:  news sources and law journals
  • Political science journals
  • Dissertation abstracts (search for "California" and "redistricting" in abstracts, and you will see a couple of Rose Institute names)
General Statistics 

Public Policy and Finance
California and General State Politics

Elections, Parties, Campaign Finance

Public Opinion

Tuesday, July 6, 2021

Insurrection, Six Months Later

 Alanna Durkin Richer and Michael Kunzelman at Associated Press:

The first waves of arrests in the deadly siege at the U.S. Capitol focused on the easy targets. Dozens in the pro-Trump mob openly bragged about their actions on Jan. 6 on social media and were captured in shocking footage broadcast live by national news outlets.

But six months after the insurrection, the Justice Department is still hunting for scores of rioters, even as the first of more than 500 people already arrested have pleaded guilty. The struggle reflects the massive scale of the investigation and the grueling work still ahead for authorities in the face of an increasing effort by some Republican lawmakers to rewrite what happened that day.

Among those who still haven’t been caught: the person who planted two pipe bombs outside the offices of the Republican and Democratic national committees the night before the melee, as well as many people accused of attacks on law enforcement officers or violence and threats against journalists. The FBI website seeking information about those involved in the Capitol violence includes more than 900 pictures of roughly 300 people labeled “unidentified.”

Ryan J.  Reilly at Huffington Post:

They call themselves sedition hunters, and they have receipts. They’re members of a loosely affiliated network of motivated individuals and pop-up volunteer organizations with names like Deep State Dogs and Capitol Terrorists Exposers that developed after the Jan. 6 attack to identify the Trump supporters who organized the Capitol riot and brutalized the law enforcement officers protecting the building.

The sedition hunters scour the web for any and all photographs, videos and posts from people at the Capitol during the Jan. 6 attack across well-known websites like Facebook, YouTube and Twitter along with lesser-used sites and apps like Rumble, Gab and Telegram. They’ve got spreadsheets, Google Docs, links, bookmarks, unlisted YouTube backups, group chats and screenshots, as Joan puts it, “coming out the rear end.” They can uncover new evidence of conduct that’ll elevate a misdemeanor trespassing case into something much more serious; find the highest-quality image of a suspect that could generate new leads through facial recognition; and compile multimedia databases that turn the Jan. 6 attack into an interactive, high-stakes and soul-crushing edition of Where’s Waldo.

 

Wednesday, June 17, 2020

How to Look Things Up and Find Things Out

Watch this clip from The Wire.  It is the best description of research, ever.

Lots of links on my homepage

The Internet Archive -- if there is a broken link to what you need, this site might help you find it.   For instance, a campaign website from the past: http://www.johnkerry.com/

Great stuff at Honnold Library -- which students usually overlook! (password required)
  • Databases
  • Nexis Uni: news sources and law journals
  • Dissertation abstracts
General Statistics and The Census

US Budget

Writing, Literature, and Film
Science, Health, and the Environment

California and General State Politics
Elections, Parties, Campaign Finance

Monday, April 27, 2020

Watch Out for Graphic Deception

Andrew McGill at Politico Nightly:
If you’ve spent time on Twitter or in front of cable television in the last month, you’ve probably seen a lot of charts.
While they’re vital in visualizing the pandemic — how else can we grasp the spread of a virus approximately 100 nanometers wide? — charts aren’t impartial. They have assumptions; they leave things out. And when they’re deployed to make rhetorical points, it can be devilishly difficult to find the lie.
Here are three things I look for when a new chart flashes across my Twitter feed: 
1. Does the unit of measurement make sense? Bad news can easily be made to look good, if you don’t actually show the bad news.



A few weeks back, the administration deployed this positive-looking chart to show how well testing was going in the U.S. Generally, charts that show an up-and-to-the-left trend mean good stuff is happening. But sharp-eyed chart-readers noted the graphic showed the growth in cumulative tests conducted — not an increase in the number of daily tests, the more important metric. That trend was much more grim, as redrawn charts showed. 
2. Has anyone monkeyed around with the scale? A common trick used to exaggerate — or minimize — a trend is to play around with y-axis. Take a look at this polling chart below:

Wow, looks like Candidate B is way ahead! Not really — the chart starts a number other than zero, exaggerating the gap. If the numbers on the left side of the chart look funky, beware.
3. Does the chart acknowledge uncertainty?







No one truly knows what’s going on. This is as true for coronavirus projections as it is for political polling, or even the fluctuations of the oil market. Good charts acknowledge this by showing uncertainty zones, or prominently displaying margins of error. Bad charts hide this stuff, and pretend to be all-knowing.
Charts help us understand this crisis. We just need to make sure we’re understanding it correctly.

Friday, January 17, 2020

Science, Engineering, America, and China

Elizabeth Redden at Inside Higher Ed:
The U.S. share of global science and technology activity has shrunk in some areas even as absolute activity has continued to grow, as China and other Asian countries have invested in science and engineering education and increased their research spending.
That’s one of the main takeaways of the "State of U.S. Science and Engineering" 2020 report, published by the National Science Board Wednesday. The report has historically been published every other year, but starting with this year's edition, the NSB is transitioning its format from a single report published every two years to a series of shorter reports issued more frequently.
...
The report found that employment in science and engineering has increased more rapidly than for the workforce overall, and now accounts for 5 percent of U.S. jobs.
Women accounted for just 29 percent of the science and engineering workforce in 2017, up from 26 percent in 2003. Underrepresented minorities made up just 13 percent of the scientific workforce in 2017, up from 9 percent in 2003 but below their share of the college-educated workforce.
Foreign-born workers account for 30 percent of all individuals employed in science and engineering-related occupations.
...
Global research and development expenditures have more than tripled since 2000, growing from $722 billion in 2000 to $2.2 trillion in 2017, fueled largely by growth in China. The U.S. and China together accounted for nearly half of all research and development spending -- 25 and 23 percent, respectively -- in 2017.
In the U.S., federal spending for research and development has increased since 2000, but the share of research and development funded by the federal government -- as opposed to businesses or other entities -- declined, from 25 percent in 2000 to 22 percent in 2017. Among higher education institutions -- which perform the largest amount of basic research, of which the federal government is the primary funder -- the share of research and development funded by federal sources declined from 57 percent in 2000 to 51 percent in 2017

Friday, October 4, 2019

Research, Fall 2019

Watch this clip from The Wire.  It is the best description of research, ever.

The Internet Archive -- if there is a broken link to what you need, this site might help you find it.  (A morbid example here: http://www.cnn.com/2006/SHOWBIZ/Music/04/05/uk.pitneydeath/index.html)

Great stuff at Honnold Library -- which students usually overlook! (password required)
  • Nexis Uni:  news sources and law journals
  • Political science journals
  • Dissertation abstracts (search for "California" and "redistricting" in abstracts, and you will see a couple of Rose Institute names)
General Statistics and The Census

California and General State Politics
Elections, Parties, Campaign Finance

Sunday, February 3, 2019

Advice About Research: TURN EVERY PAGE, WRITE DOWN EVERY NAME YOU SEE

At The New Yorker, Robert Caro recalls the best advice he ever got about research, from newspaper editor Alan Hathaway:
He didn’t look up. After a while, I said tentatively, “Mr. Hathway.” I couldn’t get the “Alan” out. He motioned for me to sit down, and went on reading. Finally, he raised his head. “I didn’t know someone from Princeton could do digging like this,” he said. “From now on, you do investigative work.”
I responded with my usual savoir faire: “But I don’t know anything about investigative reporting.”
Alan looked at me for what I remember as a very long time. “Just remember,” he said. Turn every page. Never assume anything. Turn every goddam page.” He turned to some other papers on his desk, and after a while I got up and left.

An episode of The Wire elaborated on similar advice:

Friday, October 6, 2017

Some Pointers on Research: A Session at the Rose Institute

Watch this clip from The Wire.  It is the best description of research, ever.

Great stuff at Honnold Library -- which students usually overlook! (password required)

  • Lexis-Nexis:  news sources and law journals
  • Political science journals
  • Dissertation abstracts (search for "California" and "redistricting" in abstracts, and you will see a couple of Rose Institute names)
General Statistics and The Census

California and General State Politics
Elections, Parties, Campaign Finance
  • Polling Report -- aggregation of national poll data
  • Gallup -- the best-known US pollster, delivering new numbers daily (for historical Gallup data, go to the Honnold Library menu, click "databases," then "Gallup Brain.")
  • 2016 exit poll

Wednesday, May 17, 2017

Nixon Checked Out on 12/6/73

As impeachment talks edges into the mainstream, it is worth remembering how Nixon handled it.

At Research and Politics, Matthew N. Beckmann has a terrific article titled: "Did Nixon Quit Before He Resigned?"  The abstract:
On August 9, 1974, Richard M. Nixon formally resigned the presidency; however, folklore hints Nixon informally quit fulfilling his duties well before then. As Watergate became less “a third rate burglary” than “high crimes and misdemeanors,” rumors of President Nixon’s wallowing, wandering, drinking, and mumbling swirled. Yet evidence for such assertions has been thin, and prevailing scholarship offers compelling reasons to believe Nixon’s institutional protocols overrode his individual proclivities. This study offers a new, systematic look at Nixon’s presidency by coding his public events and private interactions with top government officials during every day of his presidency. Contrary to our expectations, the results corroborate the rumors: Richard Nixon effectively quit being president well before he resigned the presidency. In fact, it turns out there was a defining moment when Nixon disengaged from his administration: on December 6, 1973, the day Gerald Ford was confirmed as Vice President.
The article makes ingenious use of the Presidential Daily Diary.  Here is the quoted description from the Richard Nixon Presidential Library & Museum:
The Daily Diary of files represents a consolidated record of the President’s activities. The Daily Diary chronicles the activities of the President, from the time he left the private residence until he retired for the day, including personal and private meetings, events, social and speaking engagements, trips, telephone calls, meals, routine tasks, and recreational pursuits. For any given meeting, telephone call, or event, the Daily Diary usually lists the time, location, persons involved (or a reference to an appendix listing individuals present), and type of event.1
 Here is how Professor Beckmann used it:
To extract the relevant details from these extraordinary records, we first distributed the Nixon Library’s Daily Diaries collection among a large team of undergraduate students, with each getting a random selection.2 The RA assigned a particular day would then scour the corresponding Diary to tally the President’s 5-plus-minute contacts—face-to-face or by phone—with the following top government officials: Chief of Staff; National Security Advisor; White House Counsel; White House Press Secretary; Treasury Secretary; Defense Secretary; Secretary of State; Speaker of the House; House Minority Leader; Senate Majority Leader; Senate Minority Leader. The result, then, was original data indicating Richard Nixon’s five-plus-minute contacts with 11 key government officials during each day of his presidency.
And a notable finding:
Drilling deeper, and to our surprise, we detected a specific day on which Richard Nixon effectively disengaged from his administration: December 6, 1973, the day Gerald Ford was sworn in as Vice President.5 Figure 3 displays President Nixon’s total weekly contacts with the aforementioned 11 key officials before and after Jerry Ford’s confirmation. In the 12 weeks before that date, Nixon averaged 8 (standard deviation = 5) contacts per day with top officials; in the 12 weeks after that date, he averaged 1 (standard deviation = 1).6 This does not mean it was Ford’s ascension per se that devastated Nixon; rather, we suspect Ford’s confirmation was merely the last straw—the point when Nixon realized his hopes for surviving Watergate were dashed.

Friday, January 27, 2017

Back When the White House Checked Facts

Trump and his aides routinely lie to the public and do not bother to check their facts.  Meredith Bohen writes at Vox:
As a research associate in the Office of Communications, I quickly learned some things about the nature of facts in politics. First, that fact-checking is in-depth work. My co-workers and I pored over speech draft after speech draft, methodically verifying that every individual factual statement in any of the president’s prepared remarks was backed up by reputable sources. We worked closely with the speechwriting and policy departments to ensure that each fact the president said represented his views in an accurate, verifiable way. If we couldn’t back it up, it had to go. We thought through the possible counterarguments that people from both sides of the aisle could make to rebut our statements, and we made sure we were on as solid ground as possible.

When I say we checked every fact, that’s not an exaggeration. Because we hunted down anything that could be debated as true or false, a page or two of remarks could take hours to comb through. Sometimes the job could feel like overkill (like when I spent half a day trying to verify the number of Bo and Sunny cookies served at the White House holiday parties). We even vetted each individual who was mentioned in remarks — even if they had been dead for a thousand years, and even if they were eighth-graders, which can really feel like overkill. But most of the time, the work of fact-checking felt like a necessary part of upholding the integrity of both President Obama and the office of the presidency as a whole.

Monday, January 9, 2017

Trump Databases

At Mashable, Marcus Gilmer writes:
First up, there's the matter of Trump's many campaign promises. After all, he's already backing off threats to prosecute Hillary Clinton over her use of a private email server.

That's where TrumpTracker comes in. Created by Viren Mohindra, it collects Trump's numerous campaign promises, cross-referenced with sourcing from major media outlets, Trump's own website, and even video of Trump's speeches.
...
With all of the subjective spin that permeates politics, from social media to news analysts, it can be difficult to track what, exactly, is the source of a particular debate or frenzied headlines.

Which is why FactBase's Trump database is such an interesting — and essential — searchable database. As the site's mission statement heralds, there are no stories or spin here, just the words used by Trump from speeches, debates, books, and tens of thousands of tweets.
...
In a similar vein, the Internet Archive has launched its own Trump database "with 700+ televised speeches, interviews, debates, and other news broadcasts related to President-elect Donald Trump."

Featuring over 520 hours of Trump-related content going back to December 2009, the archive also features an open-sourced fact-check spreadsheet that will continue to grow as the archive does.

(Reposted from Epic Journey) 

Friday, December 30, 2016

Farenthold Exposes Trump: Or, How to Do Investigative Reporting

At The Washington Post, David Farenthold tells how he exposed Trump's lies about his charitable giving.  It is a short intro on how to do investigative reporting. He combined old-fashioned document-hunting and phone-calling with 21st-century crowdsourcing.

He said that the story reminded him of his 2014 report about the federal government’s giant paperwork cave -- a huge facility where hundreds of federal employees still used paper files to compute federal retiree benefits.
In reporting jargon, I’d tried the front door: I asked to tour the mine. OPM said no. So then I went looking for windows. I sought out ex-employees, who had firsthand knowledge of the place but weren’t beholden to OPM’s desire for secrecy.
I found them. By piecing together their recollections, I got the story that the government didn’t want me to find.
Now Trump himself was the abandoned limestone mine.
If he wouldn’t tell me what he had given away, I’d try to find the answer anyway — by talking to charities with firsthand knowledge of what he had given.
He learned that Trump had illegally used $20k in Trump Foundation money to buy a portrait of himself.
I kept looking, posting details of my search to Twitter. Soon I had attracted a virtual army, ready to join the scavenger hunt. I had begun the year with 4,700 Twitter followers. By September I had more than 60,000 and climbing fast.
He did not find the specific portrait he was looking for. Then the story took a turn.
“Google ‘Havi Art Trump,’ ” said a strange voice on the phone one day, calling from the 561 area code. Palm Beach, Fla.
I did.
The Google search revealed a new portrait of Trump. This one was four feet tall, painted by Miami artist Havi Schanz. After a phone call, I confirmed that Trump had purchased it in 2014 at a charity auction run by the Unicorn Children’s Foundation. Once again, he had the Trump Foundation pay the bill.
I needed to find that portrait. I turned to my Twitter followers, putting out a photo of the new $10,000 portrait.
That was at 10:34 a.m.
By early evening I knew where it was.
“The Havi Painting was at Doral National in Miami, you can see two separate pics that tourists have taken of it,” wrote Allison Aguilar.
I’ve never met Aguilar. I learned later that she is a former HR manager who is now a stay-at-home mother in Atlanta, writing short stories on the side. Days before, looking for the $20,000 portrait, she had scoured the website for Trump’s golf resort at Doral, in Florida, scanning more than 500 user-generated photos of the resort’s rooms, restaurants and golf course.
About halfway through, she had spotted another portrait in a photo, hanging on a wall at the resort.
Then she saw my tweet, saying that I was now looking for that portrait, too.
“Oh, now that I’ve seen,” Aguilar remembered thinking.
The TripAdvisor photo she found was dated February 2016.
Was the portrait still there?
The answer was provided by another stranger.
Enrique Acevedo, an anchor at the Spanish-language network Univision, saw my tweet that night, broadcasting that Aguilar had traced the portrait to Doral. Acevedo realized that Doral was just a few blocks from the Univision studios. He booked a room for that night.
“I used points,” Acevedo said. “I didn’t want to ... spend any money on Trump’s property, so I used points.” After his newscast ended, Acevedo checked in and started quizzing the late-night cleaning crews.
“Have you seen this picture?” he asked. “They said, ‘Oh yeah, it’s downstairs.’ ”
Bingo. Acevedo found the $10,000 portrait, paid for with charity money, hanging on the wall of the resort’s sports bar.

Saturday, September 6, 2014

False Friends of the Court

At The New York Times, Adam Liptak writes that the Supreme Court is increasingly reliant on facts from amicus briefs.
But this is a perilous trend, said Allison Orr Larsen, a law professor at the College of William and Mary.
“The court is inundated with 11th-hour, untested, advocacy-motivated claims of factual expertise,” she wrote in an article to be published in The Virginia Law Review.
Some of the factual assertions in recent amicus briefs would not pass muster in a high school research paper. But that has not stopped the Supreme Court from relying on them. Recent opinions have cited “facts” from amicus briefs that were backed up by blog posts, emails or nothing at all.
Some amicus briefs are careful and valuable, of course, citing peer-reviewed studies and noting contrary evidence. Others cite more questionable materials.
Some “studies” presented in amicus briefs were paid for or conducted by the group that submitted the brief and published only on the Internet. Some studies seem to have been created for the purpose of influencing the Supreme Court.
Yet the justices are quite receptive to this dodgy data. Over the five terms from 2008 to 2013, the court’s opinions cited factual assertions from amicus briefs 124 times, Professor Larsen found.

Monday, June 30, 2014

Investigating Institutions

"Investigating Powerful Institutions: Inside and Out" is 2-page handout by NYT reporter Matt Apuzzo, from the annual Investigative Reporters and Editors conference in San Francisco.  Its insights on written sources apply to opposition research.
Read: Read everything that’s been written. Do a Nexis search for your organization, going back at least five years. If you’re looking into a large organization – Pfizer, the CIA, General Electric – narrow your search with keywords: Pfizer and sales representatives. CIA and Russia. General Electric and medical devices. Select every story that is even remotely relevant. Sort chronologically. Save it to a PDF. Put it in on all your devices and read it whenever you have a
moment – on the subway, before bed, while you’re on hold, while you’re having coffee. As you read, write down all the names you come across. This is the public history of your organization.
...
Documents: Generally, there are two kinds of documents. Revelatory documents and roadmap documents. The key email, the internal report, the explosive audit: These are revelatory documents. They are the backbone of many a great story and many a great IRE panel (so much so that I won’t spend a ton of time here on all the great ways to find those documents). But as a strategy, always be thinking: Where would it be written down, who would have it, and what would that document be called? Everything is written down somewhere.
Roadmap documents are much less sexy, but just as important. Phone directories. Organizational charts. Annual reports. Legal opinions. Flow charts. Policy documents. (Even better, drafts of policy documents!). Look for places where your organization intersects with the government, and you’ll find public records there. Is it regulated? Does it have to file paperwork with the government? Does it receive state or federal money? Does it try to influence the government? Are its facilities inspected, its real properties taxed? What does it own? Does it invent things and seek patents?
One of my favorite sources of roadmap documents is lawsuits, because of all the discovery
documents – depositions, emails, etc. – that comes with them. Even if they aren’t at all related to what I’m investigating, they give me access points and help me understand the culture. I also love divorce records and bankruptcy records.

Saturday, September 22, 2012

The Vanishing Internet

If you have tried to do historical research online, you quickly trip across broken links.  The Internet Archive is one partial solution, but Rebecca J. Rosen writes in The Atlantic that it is not enough:
There is a well-known thrill that comes from watching -- nearly in real time -- as big news unfolds on Twitter. Millions upon millions of people pass information around, celebrate it, mourn it, and discuss it. How will this whole process look to historians of the future? Will they be able to recreate and understand what it was like?
The question goes beyond the archiving of the tweets themselves (something the Library of Congress has taken a lead on), though that matters too. But the tweets are only part of the story; where their links bring readers to also needs to be preserved, and that's not happening fast enough, according to a new study (pdf) from Hany M. SalahEldeen and Michael L. Nelson at Old Dominion University in Norfolk, Virginia.
They looked at some one million tweets from six historical events over the past three years (Iranian elections, Michael Jackson's death, the H1N1 outbreak, Obama's Nobel Peace Prize, the Egyptian revolution, and the recent Syrian uprising) and found that archiving is not keeping apace with the web's fast turnover -- as time progressed, the webpages linked to became increasingly unavailable. "We estimate that after a year from publishing about 11 percent of content shared in social media will be gone," they write. "After this point, we are losing roughly 0.02 percent of this content per day." [emphasis added]