HuggingFace：pipeline为特定NLP任务直接调用

默认情况下，pipeline选择一个特定的预训练模型，该模型已为英语情绪分析进行了微调。创建分类器对象时，将下载并缓存模型。如果重新运行该命令，则将使用缓存的模型，无需再次下载该模型。

from transformers import pipeline

classifier = pipeline("sentiment-analysis")
classifier("I've been waiting for a HuggingFace course my whole life.")

输出：[{'label': 'POSITIVE', 'score': 0.9598047137260437}]

from transformers import pipeline

classifier = pipeline("zero-shot-classification")
classifier(
    "This is a course about the Transformers library",
    candidate_labels=["education", "politics", "business", "course"],
)

输出：
{'sequence': 'This is a course about the Transformers library',
 'labels': ['course', 'education', 'business', 'politics'],
 'scores': [0.9461037516593933,
  0.04552055522799492,
  0.0060350666753947735,
  0.002340571256354451]}

from transformers import pipeline

generator = pipeline("text-generation")
generator("In this course, we will teach you how to", max_length=20, num_return_sequences=3)

输出：
[{'generated_text': 'In this course, we will teach you how to manipulate some of the basics of working with and analyzing'},
 {'generated_text': 'In this course, we will teach you how to create a database structure and its parameters by writing a'},
 {'generated_text': 'In this course, we will teach you how to read and write with great care to make sure your'}]

针对需求的任务，在HuggingFace官网上搜索模型，输入到model参数

from transformers import pipeline

generator = pipeline("text-generation", model="distilgpt2")
generator(
    "In this course, we will teach you how to",
    max_length=30,
    num_return_sequences=2,
)

输出:
[{'generated_text': 'In this course, we will teach you how to use a small amount of JavaScript and JavaScript in combination with more advanced Javascript and JavaScript.'},
 {'generated_text': 'In this course, we will teach you how to use the best approach of the class: use the principles of a basic programming language like C++ and'}]

参数 top_k：控制要显示的可能性数量。

from transformers import pipeline

unmasker = pipeline("fill-mask")
unmasker("This course will teach you all about  models.", top_k=3)

输出：
[{'sequence': 'This course will teach you all about mathematical models.',
  'score': 0.1961982101202011,
  'token': 30412,
  'token_str': ' mathematical'},
 {'sequence': 'This course will teach you all about computational models.',
  'score': 0.04052715376019478,
  'token': 38163,
  'token_str': ' computational'},
 {'sequence': 'This course will teach you all about predictive models.',
  'score': 0.03301785886287689,
  'token': 27930,
  'token_str': ' predictive'}]

from transformers import pipeline

ner = pipeline("ner", grouped_entities=True)
ner("My name is Sylvain and I work at Hugging Face in Brooklyn.")

输出：
[{'entity_group': 'PER',
  'score': 0.9981693774461746,
  'word': 'Sylvain',
  'start': 11,
  'end': 18},
 {'entity_group': 'ORG',
  'score': 0.9796019991238912,
  'word': 'Hugging Face',
  'start': 33,
  'end': 45},
 {'entity_group': 'LOC',
  'score': 0.9932105541229248,
  'word': 'Brooklyn',
  'start': 49,
  'end': 57}]

请注意，此pipeline通过从提供的上下文中提取信息来工作；它不会生成答案。

from transformers import pipeline

question_answerer = pipeline("question-answering")
question_answerer(
    question="Where do I work?",
    context="My name is Sylvain and I work at Hugging Face in Brooklyn",
)

输出：
{'score': 0.6949757933616638, 'start': 33, 'end': 45, 'answer': 'Hugging Face'}

与文本生成一样，可以为结果指定max_length或min_length

from transformers import pipeline

summarizer = pipeline("summarization")
summarizer(
"""
    America has changed dramatically during recent years. Not only has the number of
    graduates in traditional engineering disciplines such as mechanical, civil,
    electrical, chemical, and aeronautical engineering declined, but in most of
    the premier American universities engineering curricula now concentrate on
    and encourage largely the study of engineering science. As a result, there
    are declining offerings in engineering subjects dealing with infrastructure,
    the environment, and related issues, and greater concentration on high
    technology subjects, largely supporting increasingly complex scientific
    developments. While the latter is important, it should not be at the expense
    of more traditional engineering.

    Rapidly developing economies such as China and India, as well as other
    industrial countries in Europe and Asia, continue to encourage and advance
    the teaching of engineering. Both China and India, respectively, graduate
    six and eight times as many traditional engineers as does the United States.

    Other industrial countries at minimum maintain their output, while America
    suffers an increasingly serious decline in the number of engineering graduates
    and a lack of well-educated engineers.

"""
)

输出：
[{'summary_text': ' America has changed dramatically during recent years . The number of engineering graduates in the U.S. has declined in traditional engineering disciplines such as mechanical, civil,    electrical, chemical, and aeronautical engineering . Rapidly developing economies such as China and India continue to encourage and advance the teaching of engineering .'}]

from transformers import pipeline

translator = pipeline("translation", model="Helsinki-NLP/opus-mt-zh-en")
translator("今天真是开心的一天！")

输出：
[{'translation_text': "It's been a happy day!"}]

目前展示的pipeline主要用于演示目的。它们是为特定任务编程的，不能执行不同的任务。

下次将介绍pipeline（）函数的内部内容以及如何自定义其行为。

Original: https://blog.csdn.net/m0_50896529/article/details/121794421
Author: 郑不凡
Title: HuggingFace：pipeline为特定NLP任务直接调用