Journal of information and
communication convergence engineering
(J. Inf. Commun. Converg. Eng.)

Automatic text summarization (ATS) shortens lengthy text to produce a summary containing the most important information within the original content. ATS plays an important role in addressing the issue of information overload and can be characterized based on its operations: homogeneity (single-document vs. multi-sourced) and rewriting approaches (extractive vs. abstractive). State-of-the-art techniques for ATS include eigen-decomposition [1], neural methods [2], and large language models [3]. However, all of these techniques require a large amount of training data to learn to identify the gist of the original content and rewrite it in an understandable format. In less-privileged languages, resources for ATS are scarce.

Compared to popular languages, publicly accessible datasets for ATS in Thai language are relatively common. Thai orthography poses challenges for language preprocessing owing to its complex nature, as it lacks explicit markers for word and sentence boundaries, making annotation a challenging task for linguists. TR-TPBS [4], ThaiSum [4], Thai-CrossSum [5], and XL-Sum [6] are datasets created for Thai ATS based on different and unattested guidelines. ATS techniques aim to shorten a given text and extract relevant information from large amounts of textual data. For Thai, ATS can be performed either extractively [7] or abstractively [8]. Although publicly available, they are not designed to evaluate different rewriting methods, and there are no annotation guidelines for dataset construction.

The dataset must be prepared prior to developing the ATS, in which three steps are applied: source observation, data collection, and annotation. The goal of this study is to construct an ATS system for economic news articles in Thai language trained on public datasets. The dataset used in this study was carefully curated from the following reliable news sources: Bangkok Turakij (https://www.bangkokbiznews.com), Prachachat Turakij (https://www.prachachat.net), and Bangkok Today (https://bangkok-today.com).

This paper describes the construction of a Thai language resource called ThEconSum, which is a corpus of ATS for Thai economics news articles. This dataset presents several language difficulties in Thai summarization. This study makes three contributions. First, a standard ATS dataset was distributed free of charge, and guidelines for data construction were elaborated. Second, we developed basic models utilizing powerful language models such as BERT and mT5. Finally, this dataset serves as a valuable resource for condensing knowledge for large-scale ATS applications in the economic domain.

The remainder of this paper is organized as follows. In Section 2, we explain the remarkable linguistic behaviors of Thai related to ATS and public ATS datasets. In Section 3, we elaborate on the construction of our dataset and annotation guidelines. In Section 4, we benchmark the baselines for the Thai ATS using our dataset and report the results. Finally, we conclude the paper in Section 6.

A. Linguistic Behaviors of Thai

Thai belongs to the analytic language family and is characterized by its adherence to subject-verb-object (SVO) word order and head-initialism. Adjectives, adverbs, function words, and syntactic structures are used to express grammatical functions like numerals, tenses, and voices without using any inflection or declension system. Because more words are employed to communicate the same notions due to semantic minimalism, a Thai phrase is often longer than an identical translation in the European language. Furthermore, in Thai orthography, word- and sentence-boundary indicators are implicit and must be implied in context.

This poses several challenges for the ATS. First, data preparation must include word tokenization and sentence-boundary identification. Second, because Thai lacks an inflection system, compound words can arise without the use of inflections, making key phrases difficult to learn. Finally, Thai sentences tend to be long and contain a significant number of function words and complex syntactic structures spanning long ranges. These linguistic characteristics must be carefully considered and properly addressed when designing and developing ATS models for Thai language.

B. Datasets for ATS

Compared to widely spoken languages, public datasets for Thai ATS are relatively limited. This is primarily because of the challenges involved in preprocessing the Thai language. In Thai orthography, words are written consecutively, whereas sentences are written with space delimiters, which causes orthographical ambiguity and makes linguistic annotation a time-consuming task.

However, there are currently four datasets available that specifically cater to Thai ATS. The datasets used were as follows:

• TR-TPBS [4] was compiled using information from ThaiRath and Thai PBS news websites. More than 333,000 articles and summary pairings produced by journalists are included in this collection.

• ThaiSum [4] is an extensive dataset designed specifically for Thai ATS. It was sourced from reliable sources, such as The Standard, Prachatai, Thai PBS, and Thai Rath. This dataset comprises a collection of more than 350,000 article-summary pairs that were meticulously compiled by journalists. On average, the article texts spanned 530 words, whereas the corresponding summaries condensed the information into a concise form of approximately 37 words. ThaiSum is a valuable resource for training and evaluating Thai summarization models.

• ThaiCrossSum Corpora [4] is an extensive dataset specifically designed for the cross-lingual summarization of Thai literature. This dataset consists of 310,926 TH2EN (Thai-to-English) articles and 310,926 TH2ZH (Thai-to-Chinese) articles.

• XL-Sum [6] stands as the largest dataset accessible for multilingual ATS. It consists of a collection of 1.35 million article-summary pairs spanning 44 different languages.

For Thai, XL-Sum contains 8,268 article-summary pairs. Table 1 lists the statistics of the aforementioned datasets. ThaiSum is the largest dataset for Thai news ATS, with approximately 530 words per article and 37 words per summary. The article parts of XL-Sum and our dataset were longer (967 and 766 words, respectively). Although the summary parts of ThaiSum and XL-Sum are comparable, our dataset is substantially larger. Note that the TR-TBPS and Thai-CrossSum datasets were not included in the table because TR-TBPS is already included in ThaiSum, and the issue statement part of Thai CrossSum is incompatible with our dataset.

Table 1 . Comparison of data statistics.

Dataset	Size	Avg. Article length (words)	Avg. Summary length (words)	Annotator
ThaiSum	350,000	530	37	Journalist
XL-Sum	*8,268	967	53	Journalist
This work	3,189	766	334	Linguist

* (only Thai).

C. Related Works on ATS

ATS methods can be classified into two main categories: extractive and abstractive. In an extractive approach, a summary is generated by identifying and selecting the most relevant and significant sentences from the original input text.

There are two approaches to ATS. The extractive approach identifies the most relevant sentences in the original input text. These sentences were then concatenated to form a summary without any further generation or rearrangement. The primary objective is to extract essential information from the input text and present it in a condensed form without modifying the original structure or content of the sentences. On the other hand, the abstractive method is based on communicating the most significant information from the input text and constructing the summary using stylistics and rephrasing.

The ATS has been widely studied and applied in various fields. ATS is becoming increasingly crucial because of the massive volume of textual materials [9]. ANN-based language models are employed to produce a summary by wordby-word prediction given an already generated context [10].

This study focuses on an abstractive approach to single-document summarization (SDS). A summary is produced from a single text document while preserving the relevant information in the original text. Natural language processing (NLP) approaches are required for an abstractive ATS approach to capture the important gists of the original text. It then rewrites them in a shorter and more understandable text [12,13].

D. Thai Text Summarization

Thai ATS has recently gained attention in the NLP community. Most of these studies used an extraction method. Based on this extraction method, Ketui and Theeramunkong [7] proposed a comprehensive strategy for summarizing multiple documents using an extractive approach. This strategy comprises three main components. The first step involves unit segmentation, which is a technique used to preprocess data by performing tasks such as breaking it down into meaningful units through word segmentation and labeling parts of speech (POS). The second step is the unit-graph formulation, which encompasses the creation of a graph representation where each unit is represented as a node with a specific weight and the relationships between these nodes are depicted as weighted links. This graph structure helps capture the interconnections and importance of the units in the text. The third step focuses on unit selection for summarization. This entails identifying the nodes and linkages that hold significant relevance in the summarization task, thereby allowing the extraction of crucial information from the original documents. By following this approach, valuable insights can be obtained by summarizing Thai texts efficiently and effectively. Nuanplord and Sodanil [14] suggested an ontology-based strategy for summarizing Thai health news. Yongkiatpanich and Wichadakul [15] suggested an extractive text-summarizing strategy that integrated ontology and graphbased methodologies. A graph representation technique that integrated the Unified Medical Language System (UMLS), a comprehensive knowledge ontology obtained from the National Library of Medicine (NLM), was employed. This graphbased approach is combined with a method called Word Mover’s Distance, which is a distance function commonly used to quantify the similarity between text documents. By leveraging UMLS and Word Mover’s Distance, the researchers aim to enhance the effectiveness of their summarization method and improve the accuracy of extracting important information from medical texts. To select the main sentences, we retrieved summaries using Google PageRank, which is a popular graph-based technique.

In the abstract method, Chaowalit and Sornil [16] proposed an ATS system for customer reviews. They evaluated their method using genuine reviews of 50 goods chosen randomly from a well-known cosmetic website. Jumpathong et al. [8] developed a deep learning-based performance study on Thai news abstractive summarization. They examined the order in which the words in the original manuscript were used to construct a summary. They also found that the text length affected the model accuracy.

The construction of ThEconSum consists of three steps: data collection, building summary guidelines, and summarizing. (1) Data Curation from News Sources: we gathered news articles from reliable sources, such as Bangkok Turakij, Prachachat Turakij, Bangkok Today, and others, ensuring their reliability. (2) Annotation Guideline Construction: We created guidelines for systematically and consistently constructing a summary for each news article. The output summary is created using extractive and abstractive approaches. The extractive method involves selecting the most salient text chunks from the original content without any alterations or modifications. In the abstractive method, the relevant information in the original text is recreated as a shorter summary using paraphrasing and stylistics. (3) Manual Summarization: Four Thai language specialists were hired to create a text summary corpus. We provided them with a summarizing guideline and an online summarizing tool, which is an online corpus editor that works anywhere. They then built a summary based on the summary guidelines.

Figure 1. Process of corpus construction

A. Data Collection

Considering the following features, we curated news articles for summary: Articles related to economics were also collected. We gathered these articles exclusively from reliable sources, such as Bangkok Turakij, Prachachat Turakij, and Bangkok Today. To narrow the keyword search, we selected economics articles related to PTT Digital Solutions Company Limited because they are involved in many economic sectors and some of the authors also have business insights. Furthermore, the selected news articles varied in length from 400 to 1000 words per document. Finally, we selected economic news articles posted between June 2021 and June 2022.

After curating the data, they were imported into a corpus annotation system. The dataset was preprocessed to remove embedded HTML tags and JavaScript code, and to determine paragraph boundaries. Paragraphs play an important role in creating summaries, as explained in the following section.

B. Summarizing Guideline

We hired four Thai language specialists (bachelor’s graduates in Thai studies) to conduct the online summarization work. Universal to all languages, the guidelines for constructing a summary for each news article are summarized in the following five steps:

Step 1 (Analyze): The article was analyzed in four consecutive parts: title, leading paragraph (most newsworthy information), organizing bridge (important details), and supporting details. These four parts were inspired by the theory of news structure (a. k. a.) the Inverted Pyramid of Journalism.

Step 2 (Divide): The supporting details are divided into topics with respect to the leading paragraph. If additional topics that were not mentioned in the lead paragraph were identified, they were marked as such.

Step 3 (Rank): Rank each sentence in the leading paragraph based on its relevance to the title. Each topic is then ranked according to its relevance to the leading paragraph. Additional topics are assigned the lowest rank. This step helps to identify the content hierarchy of the article.

Step 4 (Extract): Given the preferred range of word count, select the most relevant sentences from the leading paragraph. If the word count allowed, additional supporting details relevant to the previously extracted sentences were selected. This step provides an extractive summary of the study.

Step 5 (Rewrite): Combine relevant sentences that were previously extracted from the article using syntactic processes, such as the formation of compound and complex sentences and sentence paraphrasing. This section provides an abstractive summary of the article.

Our summary guidelines have a shallow learning curve for language specialists. In addition to learning to use the corpus annotation system, they were able to master these guidelines for several days and provide high-quality news summaries using both extractive and abstractive rewriting methods.

In our settings, we set the threshold of word counts to be in the range of 35-45% of the original content. This range was observed in a preliminary experiment in which summaries of different lengths were crafted from the same articles, and this range was voted on to yield the highest-quality summaries.

For ease of use, our corpus annotation system was equipped with specialized facilities for summary crafting. The left side of the display shows the original content, whereas the right side shows extractive and abstractive summaries in the two divided panes. Each pane has a word counter that flashes green when the summary is in the preferred range of word counts (35-45%).

C. Annotators

For the construction of a text summarization corpus, we hired four Thai language specialists to build a text summarization corpus. We have provided summary guidelines and an online summary and checking system. They then built a summary based on the summarization guidelines.

Figure 2. Corpus annotation tool

D. Summarizing Software and Output Format

For corpus construction, we offered an online tool called a news-summarizing and checking system. This system serves as an online editor for the corpus, and allows users to access and work at any location. This tool was designed to assist linguists in constructing the text summarization corpus. It shows the essential details for summarizing, that is, the original text and the word frequency in the original text.

E. Data Statistics

The ThEconSum dataset contains 3,189 documents with both extractive and abstractive summaries. The average word count of the documents was approximately 766. The average word counts of the extractive and abstractive summaries were 340 and 334, respectively. When comparing the summary with the original content, it was found to be approximately 44%.

F. Exploratory Data Analysis

Fig. 3 shows the word clouds of the most frequent words in the dataset. The most frequent words include “million baht,” “public company limited,” “expected,” and “in year.” This signifies the predominant emphasis of economic news on a company’s metrics and future predictions.

Figure 3. Word cloud of the dataset

Fig. 4 illustrates the length distribution of the content and summaries, both of which exhibit symmetrical distributions around their respective means. The content length spanned 400-1200 words, with a mean and median of approximately 770 words. By contrast, the summaries were between 200 and 500 words, with a mean of 340 words. Additionally, based on the mean and median values, extractive summaries were slightly shorter than abstractive summaries.

Figure 4. Skewed distribution of text lengths in the dataset

The ATS comprises three steps. Preprocessing: The text is processed using stop-word removal, NER, sentence segmentation, and word tokenization. Here, named entities are usually replaced by special tags to improve system accuracy. Second, summarization: the structured text is analyzed, identified with salient information, and converted into a summary. Finally, post-processing addresses language-specific challenges such as anaphora resolution and NE replacement, and forms a final textual summary.

A. Baseline Models

The dataset is divided into two sets of experiments. In both experiments, the text was not tokenized into sentences. We believe that the accuracy reflects the consistency of the summarization, as suggested by the guidelines.

In the first experiment, we trained a transformer model from scratch using our dataset and evaluated its accuracy. The default transformer configuration is used. It consists of six encoding layers and six decoding layers, and the sinusoidal position embedding with the maximum length allowed for each document is 2048 subword tokens. Word embedding had 512 dimensions. We used the parameter initialization method of Huang et al. (2020). The embedding layers for both the encoder and decoder were shared. The model consists of 120 million parameters.

Figure 5. Architecture of an abstractive text summarization

In the second experiment, we used pre-trained language models for Thai and trained the decoder using our dataset. We compared two pre-trained LLMs, mT5 and HoogBERTa, as described below.

• mT5-small: mT5 is a cross-lingual pre-trained model for multiple tasks, including machine translation and ATS. It was pre-trained with a data collection of 101 languages from Common Crawl. Due to time and resource constraints, we chose mT5-small, a smaller version with 300 million parameters, as the baseline model.

• The Pre-trained LLM + Transformer: In Fig. 6, we utilize word embeddings from HoogBERTa, a Thai RoBERTabased pre-trained model. To handle documents that exceeded a maximum length of 512 words, we divided them into 500-subword pieces. For larger documents, we extracted the features of each chunk independently. None of the pieces overlap. The final document-level representation is formed by combining the chunk-level information. In the model architecture, the sinusoidal position encoding is, and there are six encoding and six decoding layers. To accelerate convergence, we freeze the parameters of HoogBERTa and train only the decoder. To ensure compatibility with the pre-trained embedding size, the word embedding of the encoder was set to 768 dimensions. A total of 120 million parameters were used in this model.

  Figure 6. The encoder of the model, ‘Pre-trained LM + Transformer’
  

  
  

B. Experimental Setup

The ThEconSum dataset used in these experiments was published in the AI for THAI [18]. The dataset consists of 3,189 entries, where each document is attached to extractive and abstractive summaries. We followed the 80-10-10 traintest-validation split practice. To improve the meaningfulness of the subwords, we first tokenized the articles and summaries into words using the Longan library available on AI for THAI [18]. The average word counts of the documents and summaries were 766 and 340, respectively.

The AdamW optimizer [19] was used to train the decoders of each model. We set the learning rate to 1e-3 and iterated the training for 60 epochs with a batch size of two. Validation was performed every 40 iterations. When generating the text, we set the specific hyperparameters as follows:

• Non-repetition n-gram size = 2

• Width of beam search = 2

• Repetitive penalty = 1.2

• Length penalty = 0.6

C. Evaluation Metrics

Automatic Evaluation: Following the literature, the standard metric for evaluating ATS is the Recall-Oriented Understudy for Gisting Evaluation score (ROUGE) [20]. ROUGE-n compares the n-gram occurrences in a given predicted summary with those in the reference summaries. Popular metrics include ROUGE-1 (unigram), ROUGE-2 (bigram), and ROUGE-L (longest common subsequence). These metrics were also used in the experiments.

Manual Evaluation: Thai language specialists also conducted a manual evaluation for a qualitative assessment of the produced summaries. We take into account The following criteria were considered:

• Readability: When evaluating readability, it is important to ensure that the summary maintains a coherent rhetorical structure without any gaps or dangling anaphoras.

• Grammaticality: The produced sentence should be grammatical.

• Structure and Coherence: The summary should have relevant, cohesive phrases with a clear organization of the structure.

• Content coverage: Every issue discussed in the original documents should be included in the summary, together with essential details.

• Non-redundancy: Redundant text and duplications should not be included in the summary.

Each assessment criterion is scored on a scale of 1 (very poor) to 10 (exceptional).

A. Results of Automatic Evaluation

Table 2 presents a comparison of the ROUGE scores of the ATS-TH-01 (mT5-small), ATS-TH-02 (vanilla transformer), and ATS-TH-03 (Transformer + HoogBERTa) models. ATSTH-03 significantly outperformed the other models. When we replaced the mT5 model with a vanilla transformer, we noticed an improvement in the ROGUE-1 accuracy. This suggests that the basic transformer is more efficient at learning keywords than mT5. Furthermore, the accuracy increased when we replaced the pre-trained encoder with HoogBERTa. We observed a similar trend for the ROUGE-2 and ROUGEL metrics. This is because HoogBERTa is better trained in the Thai language.

Table 2 . Results on the economic news dataset.

Model Code	Abstractive Model	R-1	R-2	R-L
ATS-TH-01	mT5-small	0.3090	0.1724	0.2952
ATS-TH-02	Transformer	0.5975	0.2326	0.4813
ATS-TH-03	Transformer + HoogBERTa	0.6228	0.3889	0.5623

B. Automatic Evaluation of Summaries

The results in Table 3 indicate that the ATS-TH-03 model (Transformer + HoogBERTa) outperformed the other models in terms of the qualitative results. Compared to the standard transformer, it provides more understandable and grammatical summaries based on the pre-trained model. ATS-TH-03 (Transformer + HoogBERTa) performs better than ATS-TH-01 (mT5-small). This implies that the multilingual models are likely to underperform when sufficient training data are available.

Table 3 . Assessment of the quality of the system-generated summaries.

Criteria	ATS-TH-01		ATS-TH-02		ATS-TH-03
	Average	Average S.D.	S.D.	Average	S.D.
1	Readability	4.4	1.506	2.6	1.838	6.1	2.183
2	Grammaticality	5.8	1.135	3.7	2.003	6.6	1.647
3	Structure and Coherence	4.5	1.354	3.0	2.160	6.1	2.132
4	Content coverage	4.11 .729	2.7	1.767	5.6	2.271
5	Non-redundancy	6.9	1.101	2.8	1.932	4.8	2.658

As suggested by the structural coherence and content coverage criteria, ATS-TH-03 (Transformer + HoogBERTa) outperformed the other models. When analyzing the produced summaries, we discovered that they preserved more meaningful relevant keywords and keyphrases. This improvement can be attributed to the multihead attention mechanism and syntactic information embedded in the pre-trained model.

However, for the non-redundancy criterion, ATS-TH-01 (mT5-small) outperformed the other criteria. We assumed that this was related to our hyperparameter settings (repetition penalty = 1.2; length penalty = 0.6). These specific hyperparameters discourage the excessive use of repetitive terms. However, despite achieving a lower non-redundancy score, the resulting outputs did not exhibit human-readable characteristics, as illustrated in subsequent sections.

A. Linguistic Analysis

ATS-TH-01 (mT5-small) was unable to capture the main points of news articles. While the generated summaries may be grammatically correct and related to keywords, they tend to be disorganized and lack coherence. Compared to the other models, ATS-TH-01 produced fewer repetitions and duplications.

ATS-TH-02 (vanilla Transformer) exhibits the lowest readability. This model randomly selects text fragments from a source document and combines them into an output, resulting in an incoherent summary. Relevant keywords and keyphrases were rarely generated, whereas repetitions were frequently found.

The ATS-TH-03 (Transformer + HoogBERTa) demonstrated superior performance. It exhibits the highest level of readability and effectively covers relevant keywords. Additionally, there were significantly fewer repetitions and duplicates.

Furthermore, we investigated the correlation between document length and the ROUGE score of the ATS-TH-03 model output. We categorized the test data into three ranges based on their lengths and evaluated their individual ROUGE scores (Table 4). The summaries of news articles between 801 and 1000 words had the highest ROUGE-1, ROUGE-2, and ROUGE-L.

Table 4 . Effects of document lengths and ROUGE Scores of the ATS-TH-03 Model.

Document Length (word)	Number of Document	ROUGE Score
		R-1	R-2	R-L
400-600	103	0.6102	0.3880	0.5639
601-800	125	0.6171	0.3726	0.5462
801-1000	92	0.6444	0.4122	0.5823

B. Implications

Field generalization: Because the model was trained in a specific domain, it has the potential to be utilized for ATS in the economic arena. Our experimental results demonstrate that a general-domain pre-trained model can be fine-tuned with a relatively small domain-specific dataset, and it manifests promising quantitative and qualitative results.

Limitations: First, the stylistics of the training data directly affect the accuracy of the ATS. Because the model was trained on news articles written in formal Thai, it recognized the structure of the written language. When applied to social media data, such as Facebook and Twitter, the accuracy may decrease significantly because of informal spoken language. Second, the maximum input and output lengths significantly affect the accuracy of the ATS. For all the models, we set the maximum input length to 1,024 tokens and the maximum output length to 512 tokens. Varying these numbers may result in shorter or longer summaries; however, this does not guarantee an increase in quality gain.

This paper describes the development of a language resource for ATS, called ThEconSum, in which we address several linguistic challenges in Thai. This study makes three major contributions to the literature. First, we present a standardized dataset for the Thai ATS, accompanied by our detailed guidelines. Second, we introduce three baseline models based on language models. Finally, the dataset can be effectively employed for knowledge distillation, especially to train large-scale ATS models specifically designed for economics. For instance, a dataset can be utilized to create a domain-specific ATS system using knowledge distillation, in which a teacher-student model is employed.

Our future work will be as follows: First, we investigate the subjectivity of summarizing jobs and design a precise data-building rule. Our next step involves incorporating semantic roles similar to the 5W1H questions (who, what, when, where, why, and how) into cross-domain ATS models. This inclusion aims to enhance the accuracy and coverage of the models. Finally, we aim to select the core essence of the papers by leveraging the weak signals identified through a graph-based semantic interpretation.

This study was conducted with collaborative support from Promes Co., Ltd. (Backyard Group) as part of the Joint Research Project funded by the National Science and Technology Development Agency (NSTDA), Thailand. Additionally, we extend our gratitude to PTT Digital Solutions Co. Ltd., Thailand for generously providing public datasets for Thai ATS. The Program Management Unit for National Competitiveness Enhancement, under the Office of the National Higher Education Science Research and Innovation Policy Council in Thailand, provided financial support for data collection and construction.