A Closer Look at Generative AI

Artificial intelligence is already designing microchips and sending us spam, so what's next? Here's how generative AI really works and what to expect now that it's here.

Generative AI is an umbrella term for any kind of automated process that uses algorithms to produce, manipulate, or synthesize data, often in the form of images or human-readable text. It's called generative because the AI creates something that didn't previously exist. That's what makes it different from discriminative AI, which draws distinctions between different kinds of input. To say it differently, discriminative AI tries to answer a question like "Is this image a drawing of a rabbit or a lion?" whereas generative AI responds to prompts like "Draw me a picture of a lion and a rabbit sitting next to each other."

This article introduces you to generative AI and its uses with popular models like ChatGPT and DALL-E. We'll also consider the limitations of the technology, including why "too many fingers" has become a dead giveaway for artificially generated art.

The emergence of generative AI

Generative AI has been around for years, arguably since ELIZA, a chatbot that simulates talking to a therapist, was developed at MIT in 1966. But years of work on AI and machine learning have recently come to fruition with the release of new generative AI systems. You've almost certainly heard about ChatGPT, a text-based AI chatbot that produces remarkably human-like prose. DALL-E and Stable Diffusion have also drawn attention for their ability to create vibrant and realistic images based on text prompts. We often refer to these systems and others like them as models because they represent an attempt to simulate or model some aspect of the real world based on a subset (sometimes a very large one) of information about it.

Output from these systems is so uncanny that it has many people asking philosophical questions about the nature of consciousness—and worrying about the economic impact of generative AI on human jobs. But while all these artificial intelligence creations are undeniably big news, there is arguably less going on beneath the surface than some may assume. We'll get to some of those big-picture questions in a moment. First, let's look at what's going on under the hood of models like ChatGPT and DALL-E.

How does generative AI work?

Generative AI uses machine learning to process a huge amount of visual or textual data, much of which is scraped from the internet, and then determine what things are most likely to appear near other things. Much of the programming work of generative AI goes into creating algorithms that can distinguish the "things" of interest to the AI's creators—words and sentences in the case of chatbots like ChatGPT, or visual elements for DALL-E. But fundamentally, generative AI creates its output by assessing an enormous corpus of data on which it’s been trained, then responding to prompts with something that falls within the realm of probability as determined by that corpus.

Autocomplete—when your cell phone or Gmail suggests what the remainder of the word or sentence you're typing might be—is a low-level form of generative AI. Models like ChatGPT and DALL-E just take the idea to significantly more advanced heights.

Training generative AI models

The process by which models are developed to accommodate all this data is called training. A couple of underlying techniques are at play here for different types of models. ChatGPT uses what's called a transformer (that's what the T stands for). A transformer derives meaning from long sequences of text to understand how different words or semantic components might be related to one another, then determine how likely they are to occur in proximity to one another. These transformers are run unsupervised on a vast corpus of natural language text in a process called pretraining (that's the P in ChatGPT), before being fine-tuned by human beings interacting with the model.

Another technique used to train models is what's known as a generative adversarial network, or GAN. In this technique, you have two algorithms competing against one another. One is generating text or images based on probabilities derived from a big data set; the other is a discriminative AI, which has been trained by humans to assess whether that output is real or AI-generated. The generative AI repeatedly tries to "trick" the discriminative AI, automatically adapting to favor outcomes that are successful. Once the generative AI consistently "wins" this competition, the discriminative AI gets fine-tuned by humans and the process begins anew.

One of the most important things to keep in mind here is that, while there is human intervention in the training process, most of the learning and adapting happens automatically. So many iterations are required to get the models to the point where they produce interesting results that automation is essential. The process is quite computationally intensive. 

Is generative AI sentient?

The mathematics and coding that go into creating and training generative AI models are quite complex, and well beyond the scope of this article. But if you interact with the models that are the end result of this process, the experience can be decidedly uncanny. You can get DALL-E to produce things that look like real works of art. You can have conversations with ChatGPT that feel like a conversation with another human. Have researchers truly created a thinking machine?

Chris Phipps, a former IBM natural language processing lead who worked on Watson AI products, says no. He describes ChatGPT as a "very good prediction machine." Phipps says: 'It’s very good at predicting what humans will find coherent. It’s not always coherent (it mostly is) but that’s not because ChatGPT "understands." It’s the opposite: humans who consume the output are really good at making any implicit assumption we need in order to make the output make sense.'

Phipps, who's also a comedy performer, draws a comparison to a common improv game called Mind Meld: 'Two people each think of a word, then say it aloud simultaneously—you might say "boot" and I say "tree." We came up with those words completely independently and at first, they had nothing to do with each other. The next two participants take those two words and try to come up with something they have in common and say that aloud at the same time. The game continues until two participants say the same word.

Maybe two people both say "lumberjack." It seems like magic, but really it’s that we use our human brains to reason about the input ("boot" and "tree") and find a connection. We do the work of understanding, not the machine. There’s a lot more of that going on with ChatGPT and DALL-E than people are admitting. ChatGPT can write a story, but we humans do a lot of work to make it make sense.'

Testing the limits of computer intelligence

Certain prompts that we can give to these AI models will make Phipps' point fairly evident. For instance, consider the riddle "What weighs more, a pound of lead or a pound of feathers?" The answer, of course, is that they weigh the same (one pound), even though our instinct or common sense might tell us that the feathers are lighter.

ChatGPT will answer this riddle correctly, and you might assume it does so because it is a coldly logical computer that doesn't have any "common sense" to trip it up. But that's not what's going on under the hood. ChatGPT isn't logically reasoning out the answer; it's just generating output based on its predictions of what should follow a question about a pound of feathers and a pound of lead. Since its training set includes a bunch of text explaining the riddle, it assembles a version of that correct answer. But if you ask ChatGPT whether two pounds of feathers are heavier than a pound of lead, it will confidently tell you they weigh the same amount, because that's still the most likely output to a prompt about feathers and lead, based on its training set. It can be fun to tell the AI that it's wrong and watch it flounder in response; I got it to apologize to me for its mistake and then suggest that two pounds of feathers weigh four times as much as a pound of lead.

Why does AI art have too many fingers?

A notable quirk of AI art is that it often represents people with profoundly weird hands. The "weird hands quirk" is becoming a common indicator that the art was artificially generated. This oddity offers more insight into how generative AI does (and doesn't) work. Start with the corpus that DALL-E and similar visual generative AI tools are pulling from: pictures of people usually provide a good look at their face but their hands are often partially obscured or shown at odd angles, so you can't see all the fingers at once. Add to that the fact that hands are structurally complex—they're notoriously difficult for people, even trained artists, to draw. And one thing that DALL-E isn't doing is assembling an elaborate 3D model of hands based on the various 2D depictions in its training set. That's not how it works. DALL-E doesn't even necessarily know that "hands" is a coherent category of thing to be reasoned about. All it can do is try to predict, based on the images it has, what a similar image might look like. Despite huge amounts of training data, those predictions often fall short.

Phipps speculates that one factor is a lack of negative input: 'It mostly trains on positive examples, as far as I know. They didn't give it a picture of a seven fingered hand and tell it "NO! Bad example of a hand. Don’t do this." So it predicts the space of the possible, not the space of the impossible. Basically, it was never told to not create a seven fingered hand.'

There's also the factor that these models don't think of the drawings they're making as a coherent whole; rather, they assemble a series of components that are likely to be in proximity to one another, as shown by the training data. DALL-E may not know that a hand is supposed to have five fingers, but it does know that a finger is likely to be immediately adjacent to another finger. So, sometimes, it just keeps adding fingers. (You can get the same results with teeth.) In fact, even this description of DALL-E's process is probably anthropomorphizing it too much; as Phipps says, "I doubt it has even the understanding of a finger. More likely, it is predicting pixel color, and finger-colored pixels tend to be next to other finger-colored pixels."

Potential negative impacts of generative AI

These examples show you one of the major limitations of generative AI: what those in the industry call hallucinations, which is a perhaps misleading term for output that is, by the standards of humans who use it, false or incorrect. All computer systems occasionally produce mistakes, of course, but these errors are particularly problematic because end users are unlikely to spot them easily: If you are asking a production AI chatbot a question, you generally won't know the answer yourself. You are also more likely to accept an answer delivered in the confident, fully idiomatic prose that ChatGPT and other models like it produce, even if the information is incorrect. 

Even if a generative AI could produce output that's hallucination-free, there are various potential negative impacts:

• Cheap and easy content creation: Hopefully it's clear by now that ChatGPT and other generative AIs are not real minds capable of creative output or insight. But the truth is that not everything that's written or drawn needs to be particularly creative. Many research papers at the high school or college undergraduate level only aim to synthesize publicly available data, which makes them a perfect target for generative AI. And the fact that synthetic prose or art can now be produced automatically, at a superhuman scale, may have weird or unforeseen results. Spam artists are already using ChatGPT to write phishing emails, for instance.
• Intellectual property: Who owns an AI-generated image or text? If a copyrighted work forms part of an AI's training set, is the AI "plagiarizing" that work when it generates synthetic data, even if it doesn't copy it word for word? These are thorny, untested legal questions.
• Bias: The content produced by generative AI is entirely determined by the underlying data on which it's trained. Because that data is produced by humans with all their flaws and biases, the generated results can also be flawed and biased, especially if they operate without human guardrails. OpenAI, the company that created ChatGPT, put safeguards in the model before opening it to public use that prevent it from doing things like using racial slurs; however, others have claimed that these sorts of safety measures represent their own kind of bias.
• Power consumption: In addition to heady philosophical questions, generative AI raises some very practical issues: for one thing, training a generative AI model is hugely compute intensive. This can result in big cloud computing bills for companies trying to get into this space, and ultimately raises the question of whether the increased power consumption—and, ultimately, greenhouse gas emissions—is worth the final result. (We also see this question come up regarding cryptocurrencies and blockchain technology.)

Use cases for generative AI

Despite these potential problems, the promise of generative AI is hard to miss. ChatGPT's ability to extract useful information from huge data sets in response to natural language queries has search giants salivating. Microsoft is testing its own AI chatbot, dubbed "Sydney," though it's still in beta and the results have been decidedly mixed.

But Phipps thinks that more specialized types of search are a perfect fit for this technology. "One of my last customers at IBM was a large international shipping company that also had a billion-dollar supply chain consulting side business," he says.

Phipps adds: 'Their problem was that they couldn’t hire and train entry level supply chain consultants fast enough—they were losing out on business because they couldn’t get simple customer questions answered quickly. We built a chatbot to help entry level consultants search the company's extensive library of supply chain manuals and presentations that they could turn around to the customer.If I were to build a solution for that same customer today, just a year after I built the first one, I would 100% use ChatGPT and it would likely be far superior to the one I built. What’s nice about that use case is that there is still an expert human-in-the-loop double-checking the answer. That mitigates a lot of the ethical issues. There is a huge market for those kinds of intelligent search tools meant for experts.'

Other potential use cases include:

• Code generation: The idea that generative AI might write computer code for us has been bubbling around for years now. It turns out that large language models like ChatGPT can understand programming languages as well as natural spoken languages, and while generative AI probably isn't going to replace programmers in the immediate future, it can help increase their productivity.
• Cheap and easy content creation: As much as this one is a concern (listed above), it's also an opportunity. The same AI that writes spam emails can write legitimate marketing emails, and there's been an explosion of AI copywriting startups. Generative AI thrives when it comes to highly structured forms of prose that don't require much creativity, like resumes and cover letters.
• Engineering design: Visual art and natural language have gotten a lot of attention in the generative AI space because they're easy for ordinary people to grasp. But similar techniques are being used to design everything from microchips to new drugs—and will almost certainly enter the IT architecture design space soon enough.

Conclusion

Generative AI will surely disrupt some industries and will alter—or eliminate—many jobs. Articles like this one will continue to be written by human beings, however, at least for now. CNET recently tried putting generative AI to work writing articles but the effort foundered on a wave of hallucinations. If you're worried, you may want to get in on the hot new job of tomorrow: AI prompt engineering.

Author: Josh Fruhlinger

Source: InfoWorld 

Chatbots and their Struggle with Negation

Today’s language models are more sophisticated than ever, but they still struggle with the concept of negation. That’s unlikely to change anytime soon.

Nora Kassner suspected her computer wasn’t as smart as people thought. In October 2018, Google released a language model algorithm called BERT, which Kassner, a researcher in the same field, quickly loaded on her laptop. It was Google’s first language model that was self-taught on a massive volume of online data. Like her peers, Kassner was impressed that BERT could complete users’ sentences and answer simple questions. It seemed as if the large language model (LLM) could read text like a human (or better).

But Kassner, at the time a graduate student at Ludwig Maximilian University of Munich, remained skeptical. She felt LLMs should understand what their answers mean — and what they don’t mean. It’s one thing to know that a bird can fly. “A model should automatically also know that the negated statement — ‘a bird cannot fly’ — is false,” she said. But when she and her adviser, Hinrich Schütze, tested BERT and two other LLMs in 2019, they found that the models behaved as if words like “not” were invisible.

Since then, LLMs have skyrocketed in size and ability. “The algorithm itself is still similar to what we had before. But the scale and the performance is really astonishing,” said Ding Zhao, who leads the Safe Artificial Intelligence Lab at Carnegie Mellon University.

But while chatbots have improved their humanlike performances, they still have trouble with negation. They know what it means if a bird can’t fly, but they collapse when confronted with more complicated logic involving words like “not,” which is trivial to a human.

“Large language models work better than any system we have ever had before,” said Pascale Fung, an AI researcher at the Hong Kong University of Science and Technology. “Why do they struggle with something that’s seemingly simple while it’s demonstrating amazing power in other things that we don’t expect it to?” Recent studies have finally started to explain the difficulties, and what programmers can do to get around them. But researchers still don’t understand whether machines will ever truly know the word “no.”

Making Connections

It’s hard to coax a computer into reading and writing like a human. Machines excel at storing lots of data and blasting through complex calculations, so developers build LLMs as neural networks: statistical models that assess how objects (words, in this case) relate to one another. Each linguistic relationship carries some weight, and that weight — fine-tuned during training — codifies the relationship’s strength. For example, “rat” relates more to “rodent” than “pizza,” even if some rats have been known to enjoy a good slice.

In the same way that your smartphone’s keyboard learns that you follow “good” with “morning,” LLMs sequentially predict the next word in a block of text. The bigger the data set used to train them, the better the predictions, and as the amount of data used to train the models has increased enormously, dozens of emergent behaviors have bubbled up. Chatbots have learned style, syntax and tone, for example, all on their own. “An early problem was that they completely could not detect emotional language at all. And now they can,” said Kathleen Carley, a computer scientist at Carnegie Mellon. Carley uses LLMs for “sentiment analysis,” which is all about extracting emotional language from large data sets — an approach used for things like mining social media for opinions.

So new models should get the right answers more reliably. “But we’re not applying reasoning,” Carley said. “We’re just applying a kind of mathematical change.” And, unsurprisingly, experts are finding gaps where these models diverge from how humans read.

No Negatives

Unlike humans, LLMs process language by turning it into math. This helps them excel at generating text — by predicting likely combinations of text — but it comes at a cost.

“The problem is that the task of prediction is not equivalent to the task of understanding,” said Allyson Ettinger, a computational linguist at the University of Chicago. Like Kassner, Ettinger tests how language models fare on tasks that seem easy to humans. In 2019, for example, Ettinger tested BERT with diagnostics pulled from experiments designed to test human language ability. The model’s abilities weren’t consistent. For example:

He caught the pass and scored another touchdown. There was nothing he enjoyed more than a good game of ____. (BERT correctly predicted “football.”)

The snow had piled up on the drive so high that they couldn’t get the car out. When Albert woke up, his father handed him a ____. (BERT incorrectly guessed “note,” “letter,” “gun.”)

And when it came to negation, BERT consistently struggled.

A robin is not a ____. (BERT predicted “robin,” and “bird.”)

On the one hand, it’s a reasonable mistake. “In very many contexts, ‘robin’ and ‘bird’ are going to be predictive of one another because they’re probably going to co-occur very frequently,” Ettinger said. On the other hand, any human can see it’s wrong.

By 2023, OpenAI’s ChatGPT and Google’s bot, Bard, had improved enough to predict that Albert’s father had handed him a shovel instead of a gun. Again, this was likely the result of increased and improved data, which allowed for better mathematical predictions.

But the concept of negation still tripped up the chatbots. Consider the prompt, “What animals don’t have paws or lay eggs, but have wings?” Bard replied, “No animals.” ChatGPT correctly replied bats, but also included flying squirrels and flying lemurs, which do not have wings. In general, “negation [failures] tended to be fairly consistent as models got larger,” Ettinger said. “General world knowledge doesn’t help.”

Invisible Words

The obvious question becomes: Why don’t the phrases “do not” or “is not” simply prompt the machine to ignore the best predictions from “do” and “is”?

That failure is not an accident. Negations like “not,” “never” and “none” are known as stop words, which are functional rather than descriptive. Compare them to words like “bird” and “rat” that have clear meanings. Stop words, in contrast, don’t add content on their own. Other examples include “a,” “the” and “with.”

“Some models filter out stop words to increase the efficiency,” said Izunna Okpala, a doctoral candidate at the University of Cincinnati who works on perception analysis. Nixing every “a” and so on makes it easier to analyze a text’s descriptive content. You don’t lose meaning by dropping every “the.” But the process sweeps out negations as well, meaning most LLMs just ignore them.

So why can’t LLMs just learn what stop words mean? Ultimately, because “meaning” is something orthogonal to how these models work. Negations matter to us because we’re equipped to grasp what those words do. But models learn “meaning” from mathematical weights: “Rose” appears often with “flower,” “red” with “smell.” And it’s impossible to learn what “not” is this way.

Kassner says the training data is also to blame, and more of it won’t necessarily solve the problem. Models mainly train on affirmative sentences because that’s how people communicate most effectively. “If I say I’m born on a certain date, that automatically excludes all the other dates,” Kassner said. “I wouldn’t say ‘I’m not born on that date.’”

This dearth of negative statements undermines a model’s training. “It’s harder for models to generate factually correct negated sentences, because the models haven’t seen that many,” Kassner said.

Untangling the Not

If more training data isn’t the solution, what might work? Clues come from an analysis posted to arxiv.org in March, where Myeongjun Jang and Thomas Lukasiewicz, computer scientists at the University of Oxford (Lukasiewicz is also at the Vienna University of Technology), tested ChatGPT’s negation skills. They found that ChatGPT was a little better at negation than earlier LLMs, even though the way LLMs learned remained unchanged. “It is quite a surprising result,” Jang said. He believes the secret weapon was human feedback.

The ChatGPT algorithm had been fine-tuned with “human-in-the-loop” learning, where people validate responses and suggest improvements. So when users noticed ChatGPT floundering with simple negation, they reported that poor performance, allowing the algorithm to eventually get it right.

John Schulman, a developer of ChatGPT, described in a recent lecture how human feedback was also key to another improvement: getting ChatGPT to respond “I don’t know” when confused by a prompt, such as one involving negation. “Being able to abstain from answering is very important,” Kassner said. Sometimes “I don’t know” is the answer.

Yet even this approach leaves gaps. When Kassner prompted ChatGPT with “Alice is not born in Germany. Is Alice born in Hamburg?” the bot still replied that it didn’t know. She also noticed it fumbling with double negatives like “Alice does not know that she does not know the painter of the Mona Lisa.”

“It’s not a problem that is naturally solved by the way that learning works in language models,” Lukasiewicz said. “So the important thing is to find ways to solve that.”

One option is to add an extra layer of language processing to negation. Okpala developed one such algorithm for sentiment analysis. His team’s paper, posted on arxiv.org in February, describes applying a library called WordHoard to catch and capture negation words like “not” and antonyms in general. It’s a simple algorithm that researchers can plug into their own tools and language models. “It proves to have higher accuracy compared to just doing sentiment analysis alone,” Okpala said. When he combined his code and WordHoard with three common sentiment analyzers, they all improved in accuracy in extracting opinions — the best one by 35%.

Another option is to modify the training data. When working with BERT, Kassner used texts with an equal number of affirmative and negated statements. The approach helped boost performance in simple cases where antonyms (“bad”) could replace negations (“not good”). But this is not a perfect fix, since “not good” doesn’t always mean “bad.” The space of “what’s not” is simply too big for machines to sift through. “It’s not interpretable,” Fung said. “You’re not me. You’re not shoes. You’re not an infinite amount of things.” 

Finally, since LLMs have surprised us with their abilities before, it’s possible even larger models with even more training will eventually learn to handle negation on their own. Jang and Lukasiewicz are hopeful that diverse training data, beyond just words, will help. “Language is not only described by text alone,” Lukasiewicz said. “Language describes anything. Vision, audio.” OpenAI’s new GPT-4 integrates text, audio and visuals, making it reportedly the largest “multimodal” LLM to date.

Future Not Clear

But while these techniques, together with greater processing and data, might lead to chatbots that can master negation, most researchers remain skeptical. “We can’t actually guarantee that that will happen,” Ettinger said. She suspects it’ll require a fundamental shift, moving language models away from their current objective of predicting words.

After all, when children learn language, they’re not attempting to predict words, they’re just mapping words to concepts. They’re “making judgments like ‘is this true’ or ‘is this not true’ about the world,” Ettinger said.

If an LLM could separate true from false this way, it would open the possibilities dramatically. “The negation problem might go away when the LLM models have a closer resemblance to humans,” Okpala said.

Of course, this might just be switching one problem for another. “We need better theories of how humans recognize meaning and how people interpret texts,” Carley said. “There’s just a lot less money put into understanding how people think than there is to making better algorithms.”

And dissecting how LLMs fail is getting harder, too. State-of-the-art models aren’t as transparent as they used to be, so researchers evaluate them based on inputs and outputs, rather than on what happens in the middle. “It’s just proxy,” Fung said. “It’s not a theoretical proof.” So what progress we have seen isn’t even well understood.

And Kassner suspects that the rate of improvement will slow in the future. “I would have never imagined the breakthroughs and the gains we’ve seen in such a short amount of time,” she said. “I was always quite skeptical whether just scaling models and putting more and more data in it is enough. And I would still argue it’s not.”

Date: June 2, 2023

Author: Max G. Levy

Source: Quanta Magazine

Content Copyright Clash: New York Times' Legal Battle with Microsoft and OpenAI  

The New York Times Co. sued Microsoft Corp. and OpenAI Inc. for using its content to help develop artificial intelligence services, in a sign of the increasingly fraught relationship between the media and a technology that could upend the news industry.

The technology firms relied on millions of copyrighted articles to train chatbots like OpenAI’s ChatGPT and other AI features, allegedly causing billions of dollars in statutory and actual damages, according to a lawsuit filed in New York on Wednesday. The Times didn’t specify its monetary demands.

OpenAI has faced criticism for scraping text widely from the web to train its popular chatbot since it debuted a year ago. While it has been sued by prominent authors, this is the first challenge to its practices by a major media organization. The startup has sought licensing deals with publishers, much like Alphabet Inc.’s Google and Meta Platforms Inc.’s Facebook have done in recent years. The Times’ lawsuit said the publisher reached out to Microsoft and OpenAI in April and couldn’t reach an amicable solution.

“We respect the rights of content creators and owners and are committed to working with them to ensure they benefit from AI technology and new revenue models,” an OpenAI spokesperson said in a statement. “Our ongoing conversations with the New York Times have been productive and moving forward constructively, so we are surprised and disappointed with this development.” Microsoft declined to comment.

In July, OpenAI signed an agreement with the Associated Press to access some of the news agency’s archives. OpenAI cut a three-year deal in December with Axel Springer SE to use the German media company’s work for an undisclosed sum.

“We’re hopeful that we will find a mutually beneficial way to work together, as we are doing with many other publishers,” OpenAI’s spokesperson said Wednesday.

OpenAI has been the target of multiple lawsuits from content producers complaining that their work has been improperly used for AI training. The company faces class actions from cultural figures including comedian Sarah Silverman, Game of Thrones author George R.R. Martin, and Pulitzer-winning author Michael Chabon.

The cases are still in their early stages and could take years to fully resolve. A judge in San Francisco earlier this month hinted at trimming Silverman’s copyright lawsuit against OpenAI. The judge had already narrowed a similar Silverman suit against Meta.

New Financing

OpenAI is currently in talks with investors for new financing at a $100 billion valuation that would make it the second-most valuable US startup, Bloomberg News reported last week.

Microsoft is OpenAI’s largest backer and has deployed the startup’s AI tools in several of its products. In the lawsuit, the New York Times alleged Microsoft copied the newspaper’s articles verbatim for its Bing search engine and used OpenAI’s tech to boost its value by a trillion dollars.

Microsoft’s share price has risen 55% since ChatGPT debuted in November 2022, increasing its market capitalization to $2.8 trillion. Shares were little changed Wednesday, closing at $374.07 in New York.

“If Microsoft and OpenAI want to use our work for commercial purposes, the law requires that they first obtain our permission,” a New York Times spokesperson said in an emailed statement. “They have not done so.”

Date: January 2, 2024

Author: Mark Bergen

Source: Bloomberg

Generative AI Market Expected to Grow to $1.3 trillion in Coming Ten Years

The release of consumer-focused artificial intelligence tools such as ChatGPT and Google’s Bard is set to fuel a decade-long boom that grows the market for generative AI to an estimated $1.3 trillion in revenue by 2032 from $40 billion last year. 

The sector could expand at a rate of 42% over ten years — driven first by the demand for infrastructure necessary to train AI systems and then the ensuing devices that use AI models, advertising and other services, according to a new report by Bloomberg Intelligence analysts led by Mandeep Singh. 

“The world is poised to see an explosion of growth in the generative AI sector over the next ten years that promises to fundamentally change the way the technology sector operates,” Singh said in a statement Thursday. “The technology is set to become an increasingly essential part of IT spending, ad spending and cybersecurity as it develops.”

Demand for generative AI has boomed worldwide since ChatGPT’s release late last year, with the technology poised to disrupt everything from customer service to banking. It uses large samples of data, often harvested from the internet, to learn how to respond to prompts, allowing it to create realistic-looking images and answers to queries that appear to be from a real person. 

Amazon.com Inc.’s cloud division, Google parent Alphabet Inc., Nvidia Corp. and Microsoft Corp., which has invested billions of dollars in OpenAI, are likely to be among the biggest winners from the AI boom, according to the report. 

The largest driver of revenue growth from generative AI will come from demand for the infrastructure needed to train AI models, according to Bloomberg Intelligence’s forecasts, amounting to an estimated $247 billion by 2032. The AI-assisted digital ads business is expected to reach $192 billion in annual revenue by 2032, and revenue from AI servers could hit $134 billion, the report said.

Investors, meanwhile, took a pause from their obsession with all things AI on Thursday. The software firm C3.ai fell as much as 24% in New York, extending Wednesday’s 9% decline following a disappointing sales outlook. 

Chipmaker Nvidia, which has emerged as Wall Street’s biggest AI bet, resumed its rally, rising 3.3%. Its shares have soared by 28% since May 24 and the Silicon Valley firm briefly reached a $1 trillion valuation this week. 

Date: June 2, 2023

Author: Jake Rudnitsky 

Source: Bloomberg

Uncovering the Pitfalls of Market Research: Identifying Bad Data

Market research is a critical tool for businesses to gain insights into the competitive landscape and make informed strategic decisions. However, the quality of data collected is crucial for overall success. Bad data can lead to erroneous conclusions, wasted resources, and potentially disastrous business outcomes. 

Unfortunately, bad data is ubiquitous on the internet, and it’s very easy for unqualified content creators to publish faulty business information that looks attractive and persuasive. Robert Granader, Founder and CEO of MarketResearch.com, explains the problem this way: “On a Google results page, where many business decisions are made, an unsuspecting consumer doesn’t know if you are a PhD who spent 400 hours analyzing a market or a recent graduate putting together numbers from a press release.”

True expertise can be difficult to gauge online, and flawed data can be repeated, amplified, spun, and skewed until it is embedded in the psyche and becomes a piece of accepted industry folklore. 

Maintaining a healthy dose of skepticism is a good first step, but differentiating between good and bad data can be a challenging and time-consuming task, particularly for those with limited experience, as it requires critical thinking and evaluation skills, as well as the ability to identify credible sources and validate information.

In this article, we turn to long-time Industry Analyst Gleb Mytko from MarketResearch.com’s Freedonia Group for specific advice on navigating the thorny challenge of data quality. Based on his 10+ years of experience producing authoritative market research on everything from motorcycles to mining equipment, Mytko explains where bad data comes from and what red flags to watch out for during the research process.

What Is Bad Data?

Data is vulnerable to human error, technical glitches, bias, and intentional manipulation. Even reliable sources can have significant issues if you take the information at face value without understanding how the projections were estimated. While a variety of issues are at play, the most common hallmarks of bad data, according to Mytko, include:

• Data developed using flawed information or improper assumptions
• Data based on a faulty methodology
• Data compromised by human error
• Data that seems reasonable from one perspective but does not line up with what is known about related fields
• Data that isn’t consistent over time
• Data that contradicts reliable sources without explaining why
• Data that is unclear about its scope and can be easily misinterpreted

Data found on the internet can be faulty, as well as research found in large, hefty reports produced by low-quality research firms. If you are relying on sources that value quick answers over accuracy, you may have to wade through segmentation errors, insufficient analysis, out-of-date assumptions, and information that is out of context. 

Even worse, if you are using generative AI models such as ChatGPT, the information you receive may sound logical and look compelling but be completely fabricated and have no basis in reality. AI developers have dubbed this issue “AI hallucination.” As has been widely reported, ChatGPT has a tendency to invent phony anonymous sources and make up direct quotations, names, and dates, so it’s not exactly a fact checker’s dream.   

Whatever the source of information, always pause to consider if the data makes sense. “If the data is not consistent with historical trends and doesn’t line up with what we know about related fields, it is likely to have issues,” Mytko states. 

What Key Factors Contribute to Bad Data?

When it comes to producing accurate market research, longevity in the field and experience matter. Analysts who specialize in one industry for a long time have the historical perspective to put current developments in context and better predict where the industry is headed next. In contrast, analysts with limited experience are more likely to overlook something or make mistakes, and they may lack the necessary knowledge to work with the data. 

When knowledge, training, and experience are lacking, the quality of research may be hindered by several different stumbling blocks:

• Taking too narrow of a perspective and not accounting for all relevant factors
• Failing to understand what drives the trends in the data and overlooking historical patterns and developments
• Lacking an understanding of the scope of the data
• Not having a comprehensive and multidimensional review process 
• Producing technical errors and other oversights
• Overlooking or missing a key data point or source that contradicts your data
• Not updating or improving the data series over long periods of time 

If reliable and actionable research is a priority, analysts should not work in a bubble drawing their own conclusions. Instead, take a team approach to quality control and have layers of review to ensure everything makes sense and is consistent. Multiple sets of eyes should be in place to catch technical errors and cross-check findings.

“A comprehensive and multidimensional review process is essential for developing high-quality data, as is taking a long-term perspective and consulting a wide range of sources,” Mytko advises. 

Research firms such as The Freedonia Group use a team of editors, economists, and managers all working together to produce high-quality market research reports. In addition, analysts specialize in specific industry verticals so they become familiar with the landscape and how it changes during various business cycles. These practices help ensure quality research.

What Are Some Examples of Bad Data?

Bad data is often sneaky and can take many forms. As the examples below illustrated, It is important to carefully consider the sources and scope of data to ensure that it is accurate and properly applied.

Overhyped Predictions About New Technologies 

It’s all too easy to forget that we live in a world that’s awash in “click-bait” headlines designed to capture attention. Sensationalist predictions often accompany new technologies, such as electric vehicles or automated equipment.

“Data issues are common in new fields that are developing rapidly because there is often a lack of reputable data sources and consensus,” Mytko explains. “Instead, you frequently encounter sources with eye-catching headlines that offer little published data to back up their conclusions and don’t explain their methodology.”

For example, a source may assert that “50% off all buses sold in the U.S. will be electric” by a certain date, but what they are really talking about is transit buses, which is a smaller scope of the market. The source may not consider how feasible this data is, or whether enough electric buses will even be made available. Most likely, the author never researched which companies offer electric buses in the U.S. or how many models exist.

Generalizations That Overlook Regional Differences

Understanding technological developments in other regions of the world can be complex as well. For example, Mytko traveled to India and saw the challenges facing the electric grid firsthand. “Then I read an article that says tons of farmers will use electric and hybrid cars in the country in the next five years,” he says. These types of unrealistic predictions may be based on government announcements, or marketing hype.

Inconsistent Categorization

Media publicity about disruptive new technologies may be overblown, but information about other major industries can also be misconstrued even within reliable sources. Although U.S. government data is often considered the gold standard, it too can be wrong and give a false impression of reality. For example, if you aren’t aware that the government changed what products are assigned to specific NAIC codes by the US International Trade Commission, you might have a skewed view of import trends in a specific category from year to year. 

Data with Scope Issues

Along these lines, keep in mind that good data can be “bad data” if you do not have a clear understanding of its scope. You need to know what is included in the data. For example, does the data focus on certain product types, specific market segments, pricing levels, or geographic regions? If this information isn’t clear, the data can easily be misinterpreted or improperly used. To get a proper apples-to-apples comparison, a researcher must always be sure that the data they are looking at matches the scope of what they are thinking about.

How Can You Identify Bad Data?

Even if you are not an expert in the field, or you are studying an unfamiliar market, keep these considerations in mind to help identify bad data:

• Common sense: if something doesn’t seem right, it probably isn’t. 
• Does this data line up with what we know about historical trends?
• Is the data in line with what we know about related fields? 
• Is the data incomplete? Are there issues with the methodology? Is the scope clear?
• Is the data actionable? Does it use standard units that are possible to cross-check with other sources? Does it provide sufficient information?

By watching out for inaccurate, misleading, or incomplete data, businesses can avoid pitfalls and make better informed decisions. Relying on multiple sources, employing well-trained experienced analysts, developing a rigorous review process, and partnering with reputable market research firms that follow these same practices can also go a long way in ensuring high-quality market data. 

Author: Sarah Schmidt

Source: Market Research Blog 

US Voices Concerns over EU AI Regulations

The US warned the European Union that its proposed law to regulate artificial intelligence would favor companies with the resources to cover the costs of compliance while hurting smaller firms, according to previously undisclosed documents.The US analysis focuses mostly on the European Parliament version of the AI Act, which includes rules on generative AI. Some rules in the parliament law are based on terms that are “vague or undefined,” according to the documents, which were obtained by Bloomberg News.

The analysis is Washington’s most detailed position on the EU legislation that could set the tone for other countries writing rules for AI. One US concern is that the European Parliament focuses on how AI models are developed, whereas the US would prefer an approach that focuses on the risk involved in how these models are actually used. The analysis warns that EU regulations risk “dampening the expected boost to productivity and potentially leading to a migration of jobs and investment to other markets.”

The new rules would also likely hamper “investment in AI R&D and commercialization in the EU, limiting the competitiveness of European firms,” because training large language models is resource-intensive, it said.The US State Department feedback, including a line-by-line edit of certain provisions in the law, was shared with European counterparts in recent weeks, according to people familiar with the matter who asked not to be identified discussing private documents.

One of the people said the comments were offered in the spirit of cooperation and alignment of values. Some of the US concerns have been echoed by EU member countries in response to the European Parliament version, the person said.The State Department and the European Commission declined to comment.

The EU Parliament’s AI Act, which lawmakers voted on in June, would require more transparency about the source material used to train the large language models that underpin most generative AI products. That vote cleared the way for negotiations among parliament, the European Commission and member states, and officials hope to have a deal by the end of the year for the final rules.

The US analysis is in keeping with the State Department’s calls for a more hands-off approach to the technology so as not to stifle innovation. Secretary of State Antony Blinken objected to a number of the EU Parliament’s proposals to control generative AI during a meeting with commission officials in Sweden at the end of May. At the same time, Washington has given mixed messages to EU policymakers about its views on regulation. While the US pushedback when the commission first proposed the AI Act in 2021, some American officials have begun to view mandatory rules more favorably as AI developers and ethicists warn about the possible harms from the technology.

Aaron Cooper, head of global policy at BSA The Software Alliance, a trade group that has engaged with both US and EU officials regarding AI regulation, said it’s important for countries’ AI rules to agree on basics, including definitions. “The most important thing that the Biden administration can do is continue to have a good candid conversation with their European counterparts about what the objectives are for AI policy,” Cooper said.

While the EU is pressing ahead with the AI Act, it is still debating questions about how to regulate the building blocks of the technology, known as foundation models, and general purpose AI. Some nations worry that over-regulating the technology will make Europe less competitive. After OpenAI Inc. introduced ChatGPT and ignited a boom in generative AI last year, the European Parliament added rules that explicitly target the technology.

Previous versions of the EU’s AI Act followed risk-based focus favored by the US for AI regulation, which was also the approach laid out in a framework released earlier this year by the Commerce Department’s National Institute of Standards and Technology.

Date: October 6, 2023

Author: Peter Martin, Jillian Deutsch, and Anna Edgerton

Source: Bloomberg