Menyiapkan data pelatihan teks untuk analisis sentimen
Tetap teratur dengan koleksi
Simpan dan kategorikan konten berdasarkan preferensi Anda.
Halaman ini menjelaskan cara menyiapkan data teks untuk digunakan dalam set data Vertex AI
untuk melatih model analisis sentimen.
Data pelatihan analisis sentimen terdiri dari dokumen yang dikaitkan
dengan nilai sentimen yang menunjukkan sentimen konten. Misalnya,
Anda mungkin memiliki tweet tentang domain tertentu, seperti perjalanan
udara. Setiap tweet dikaitkan dengan nilai sentimen yang menunjukkan apakah
tweet tersebut positif, negatif, atau netral.
Persyaratan data
Anda harus memberikan minimal 10 dokumen pelatihan, tetapi totalnya tidak lebih dari
100.000.
Nilai sentimen harus berupa bilangan bulat antara 0 hingga 10. Nilai sentimen maksimumnya berdasarkan
pilihan Anda. Misalnya, jika ingin mengidentifikasi apakah sentimen
tersebut berupa negatif, positif, atau netral, Anda dapat memberi label pada data pelatihan
dengan skor sentimen 0 (negatif), 1 (netral), dan 2 (positif). Skor
sentimen maksimum untuk set data ini adalah 2. Jika ingin menangkap lebih banyak
perincian, seperti sentimen lima tingkat, Anda dapat memberi label dokumen dari
0 (paling negatif), hingga 4 (paling positif).
Anda harus menerapkan setiap nilai sentimen tersebut ke minimal 10 dokumen.
Nilai skor sentimen harus berupa bilangan bulat secara berurutan, dimulai dari nol. Jika
skor Anda tidak lengkap, atau tidak memulainya dari nol, petakan ulang skor Anda menjadi
bilangan bulat secara berurutan, dimulai dari nol.
Anda dapat menyertakan inline dokumen, atau mereferensikan file TXT yang berada di dalam
bucket Cloud Storage.
Praktik terbaik untuk data teks yang digunakan untuk melatih model AutoML
Rekomendasi berikut ini berlaku untuk set data yang digunakan untuk melatih
model AutoML.
Berikan setidaknya 100 dokumen per nilai sentimen.
Gunakan jumlah dokumen yang seimbang untuk setiap skor sentimen. Memiliki lebih banyak
contoh untuk skor sentimen tertentu dapat menimbulkan bias terhadap model tersebut.
File input
Jenis file input untuk analisis sentimen dapat berupa JSON Lines, atau CSV.
[[["Mudah dipahami","easyToUnderstand","thumb-up"],["Memecahkan masalah saya","solvedMyProblem","thumb-up"],["Lainnya","otherUp","thumb-up"]],[["Sulit dipahami","hardToUnderstand","thumb-down"],["Informasi atau kode contoh salah","incorrectInformationOrSampleCode","thumb-down"],["Informasi/contoh yang saya butuhkan tidak ada","missingTheInformationSamplesINeed","thumb-down"],["Masalah terjemahan","translationIssue","thumb-down"],["Lainnya","otherDown","thumb-down"]],["Terakhir diperbarui pada 2025-09-04 UTC."],[],[],null,["# Prepare text training data for sentiment analysis\n\n| Starting on September 15, 2024, you can only customize classification, entity extraction, and sentiment analysis objectives by moving to Vertex AI Gemini prompts and tuning. Training or updating models for Vertex AI AutoML for Text classification, entity extraction, and sentiment analysis objectives will no longer be available. You can continue using existing Vertex AI AutoML Text models until June 15, 2025. For a comparison of AutoML text and Gemini, see [Gemini for AutoML text users](/vertex-ai/docs/start/automl-gemini-comparison). For more information about how Gemini offers enhanced user experience through improved prompting capabilities, see [Introduction to tuning](/vertex-ai/generative-ai/docs/models/tune-gemini-overview). To get started with tuning, see [Model tuning for Gemini text models](/vertex-ai/generative-ai/docs/models/tune_gemini/tune-gemini-learn)\n\nThis page describes how to prepare text data for use in a Vertex AI\ndataset to train a sentiment analysis model.\n\nSentiment analysis training data consists of documents that are associated\nwith a sentiment value that indicates the sentiment of the content. For\nexample, you might have tweets about a particular domain such as air\ntravel. Each tweet is associated with sentiment value that indicates if the\ntweet is positive, negative, or neutral.\n\nData requirements\n-----------------\n\n- You must supply at least 10, but no more than 100,000, total training documents.\n- A sentiment value must be an integer from 0 to 10. The maximum sentiment value is your choice. For example, if you want to identify whether the sentiment is negative, positive, or neutral, you can label the training data with sentiment scores of 0 (negative), 1 (neutral), and 2 (positive). The maximum sentiment score for this dataset is 2. If you want to capture more granularity, such as five levels of sentiment, you can label documents from 0 (most negative) to 4 (most positive).\n- You must apply each sentiment value to at least 10 documents.\n- Sentiment score values must be consecutive integers starting from zero. If you have gaps in scores or don't start from zero, remap your scores to be consecutive integers starting from zero.\n- You can include documents inline or reference TXT files that are in Cloud Storage buckets.\n\nBest practices for text data used to train AutoML models\n--------------------------------------------------------\n\nThe following recommendations apply to datasets used to train\nAutoML models.\n\n- Provide at least 100 documents per sentiment value.\n- Use a balanced number of documents for each sentiment score. Having more examples for particular sentiment scores can introduce bias into the model.\n\nInput files\n-----------\n\nInput file types for sentiment analysis can be JSON Lines or CSV. \n\n### JSON Lines\n\nThe format, field names, value types for JSON Lines files are determined\nby a schema file, which are publicly accessible YAML files.\n\nYou can download the schema file for sentiment analysis from the\nfollowing Cloud Storage location: \n\n[gs://google-cloud-aiplatform/schema/dataset/ioformat/text_sentiment_io_format_1.0.0.yaml](https://storage.cloud.google.com/google-cloud-aiplatform/schema/dataset/ioformat/text_sentiment_io_format_1.0.0.yaml)\n\n**JSON Lines example**\n\nThe following example shows how you might use the schema to create your\nown JSON Lines file. The example includes line breaks for readability.\nIn your JSON Lines files, include line breaks only after each document. The\n`dataItemResourceLabels` field specifies, for example, [ml_use](/vertex-ai/docs/general/ml-use) and is\noptional. \n\n```\n{\n \"sentimentAnnotation\": {\n \"sentiment\": number,\n \"sentimentMax\": number\n },\n \"textContent\": \"inline_text\",\n \"dataItemResourceLabels\": {\n \"aiplatform.googleapis.com/ml_use\": \"training|test|validation\"\n }\n}\n{\n \"sentimentAnnotation\": {\n \"sentiment\": number,\n \"sentimentMax\": number\n },\n \"textGcsUri\": \"gcs_uri_to_file\",\n \"dataItemResourceLabels\": {\n \"aiplatform.googleapis.com/ml_use\": \"training|test|validation\"\n }\n}\n```\n\n### CSV\n\nEach line in a CSV file refers to a single document. The following\nexample shows the general format of a valid CSV file. The [ml_use](/vertex-ai/docs/general/ml-use) column\nis optional. \n\n```\n [ml_use],gcs_file_uri|\"inline_text\",sentiment,sentimentMax\n \n```\n\nThe following snippet is an example of an input CSV file. \n\n```\n test,gs://path_to_file,sentiment_value,sentiment_max_value\n test,\"inline_text\",sentiment_value,sentiment_max_value\n training,gs://path_to_file,sentiment_value,sentiment_max_value\n validation,gs://path_to_file,sentiment_value,sentiment_max_value\n \n```"]]
