State of the art: Identification of previous empirical studies

In this section, a brief chronological overview of the main prior empirical references concerning both the Spanish and international contexts will be made. Beginning with our immediate environment, Caffarel-Serra, Ortega-Mohedano and Gaitán-Moya (2017) focus on the analysis of a representative sample of 288 documents relating to the period 2007-2014: 239 doctoral theses and 49 research projects, reaching the conclusion that 60.71% of the papers are confined to the study of mass media (49.45% traditional versus 11.26% digital).

Along these lines, Goyanes, Rodríguez-Gómez and Rosique-Cedillo (2018) record 3,653 articles published in the 11 leading Spanish journals from 2005 to 2015. Among its many findings, the prevalence of journalism (press and journalistic practices), audiovisual communication (film and television) and studies on audiences and receivers are noteworthy, as they all account for 51% of scientific production. Based on the data provided by some authors and others, we postulate that:

• H1: traditional media will have a greater role than digital media.

In one of their latest studies, Martínez-Nicolás, Saperas and Carrasco-Campos (2019) disclose the findings of a content analysis conducted on a large sample of 1,098 articles from the leading communication journals in Spain during the 1990-2014 period, allowing them to trace different research evolution timelines. As the main result of their meticulous work, it is worth noting that almost 80% of the articles constituted empirical research, while 18% were theoretical-conceptual and only 2% methodological. Thus, we contend that:

• H2: the works in the sample will exhibit a pronounced empirical approach.

At the international level, Bryant and Miron's study (2004) includes a sample of 1,806 articles published in the journals “Journalism and Mass Communication Quarterly”, “Journal of Communication” and “Journal of Broadcasting and Electronic Media” between 1956 and 2000. After their inquiry, they declare that Framing Theory is the most prominent, followed closely by Agenda Setting and Cultivation Theory. Therefore, it seems coherent to pose that:

• H₃: Framing Theory will be the most recurring theoretical corpus.

For their part, Potter and Riddle (2007) examine 962 articles from 16 high-impact journals in the period 1993-2005, concluding that 71.4% of the studies reviewed use quantitative methods −where the survey with 32% and the experiment with 29% prevail− and 15.4% qualitative techniques. This logic prompts a new hypothesis, as well as an intimately associated research question:

• H₄: quantitative methods will have a greater presence than qualitative methods.

• RQ1: What kind of samples will the authors of empirical papers use?

With regard to empirical deployment, and although they are not mutually exclusive, the debate between quantitative and qualitative traditions is no less evocative.

Gómez-Rodríguez, Morrell and Gallo-Estrada (2017) focused on the journal “Comunicación y Sociedad”, a leading publication in Mexico. They evaluated a total of 209 papers assigned to the 2004-2016 cycle and identified some recurrent themes, which fall under the labels of: sociocultural environment (43.6%), academic (24.9%), socioeconomic (16.7%) and sociopolitical (14.8%). Therefore, and in relation to the papers pertaining to our study, we ask ourselves:

• RQ2: What topics will be addressed most frequently?

Finally, Walter, Cody and Ball-Rokeach (2018) aim to dissect 1,574 articles published in “Journal of Communication” from 1951 to 2016, performing a longitudinal analysis and comparing stages. In the most recent one, from 2010 to 2016, they found that the audience (69.3%) was the main player in the studies as opposed to the message (19.3%), the source and the policies (both with 4.6%). On the other hand, the dominant research paradigm was positivism (87.5%), well above critical and cultural −both of which were evidenced in 4.6% of studies− and rhetorical (3%). Consequently, two new hypotheses emerge:

• H5: the main object of study will be the audience.

• H6: the dominant paradigm to which studies adhere will be positivist.

Next, the method used in the study will be described, supported by all the preceding empirical initiatives.

Objective and sample

The purpose of the study was to outline the state of communication research in the Hispanic sphere through the examination of academic papers. These works, which have constituted the units of analysis, have been grouped ex post to identify, in a statistically significant way, patterns of publication or clusters, also compared, according to their impact factor and geographical origin of the journals where they were included (Spain or Latin America).

For this purpose, a content analysis was performed for being a systematic, objective and quantitative method (Riffe, Lacy, & Fico, 2014; Wimmer & Dominick, 2011), commonly applied to the study of academic texts, as we have seen in the previous section. Initially, one of the key considerations in any content analysis lies in designing a sampling plan (Igartua, 2006) which, in this case, was “multi-stage” (Neuendorf, 2016). Therefore, and in a first phase, the journals selected were those with the greatest impact in 2017 −the latest year for which data are available− in the international database “SJR-Scopus” in the category of communication (www.scimagojr.com). It was determined that journals had to rank in the first two quartiles in order to be rated as high-impact, resulting in a total of seven titles, sorted according to their position in the ranking: “Comunicar” (Q1), “El Profesional de la Información” (Q1), “Communication & Society” (Q2), “Revista Latina de Comunicación Social” (Q2), “Cuadernos.info” (Q2), “Comunicación y Sociedad” (Q2) and “Palabra Clave” (Q2). Likewise, and from 2017 onwards, it was deemed reasonable to go back five years, to 2013, in order to give the sample a certain time perspective.

In short, all those articles were stored −except editorials and reviews− contained in the websites of the seven journals during the period in question, generating a sample of n=1,548 articles. This figure represents 48.77% of the universe of published works (N=3,174) in each and every one of the Spanish and Latin American journals that were indexed in SJR-Scopus in this five-year period, which represented a margin of error of ~1.8% for a 95% confidence interval.

Categories of analysis, coding and reliability

As a guide for the examination of this representative corpus of analysis, and based on other previous studies (Barranquero & Limón, 2017; Caffarel-Serra, Ortega-Mohedano, & Gaitán-Moya, 2017; Gómez-Rodríguez, Morrell, & Gallo-Estrada, 2017; Goyanes, Rodríguez-Gómez, & Rosique-Cedillo, 2018; Martínez-Nicolás & Saperas, 2016; Martínez-Nicolás, Saperas, & Carrasco-Campos, 2019; Saperas & Carrasco-Campos, 2018; Walter, Cody, & Ball-Rokeach, 2018), a codebook was prepared consisting of the following nominal polychotomous variables, along with their reliability indicator:

1) Type of article (α_k =0.92): 1=empirical, 2=theoretical/essayistic, or 3=methodological.

2) In the case of empirical work, what method (α_k =0.83) is used? (Table 1).

3) In the case of empirical work, what type of sample (α_k =0.84) is utilized?: 0=non-empirical work, 1=probability sample, or 2=non-probability sample.

4) Theory (α_k =0.70) that provides a conceptual basis for the study (Table 1).

5) Main object of study (α_k =0.93): 1=source, 2=message, 3=audience, or 4=policies/structure.

6) Main means of communication or documentary support (α_k =0.87) in the article (Table 2).

7) General topic (α_k =0.87) of the work (Table 2).

8) Paradigm (α_k =0.96) where research is framed (Walter, Cody, & Ball-Rokeach, 2018): 1=positivist: study supported by empirical assumptions and verifiable hypotheses, using quantitative or mixed methods; 2=cultural: qualitative study about the everyday practices that create and sustain culture; 3=critical: study focused on questions of power, political economy, status quo and social structure; or 4=rhetorical: study that conceives communication as the practical art of discourse.

In total, 8 multi-categorical variables in addition to those used to identify the unit of analysis; that is, the number of the article, its publication year and the journal where it appears. Likewise, the SJR impact factor of the journals during the five years examined was collected, assigning to each unit of analysis the average impact factor of the journal in the year in which it appeared. This parameter, which acted as an independent variable, was of great help in profiling the types of articles (or clusters) resulting from the processing of the results in more detail. Finally, the data collection, which ran from September 3 to December 21, 2018, involved a team of two coders. After this process, and in order to check the reliability of their work, a random subsample was selected from ~10% of the cases, which both coders analyzed. The statistical parameter used for the reliability calculation was the Krippendorff alpha (Krippendorff, 2011; 2017), found by using the “Kalpha macro” (Hayes & Krippendorff, 2007) for SPSS (version 24). As can be seen above, the reliability of the eight variables was very satisfactory, while the average rose to M (α_k)=0.87 (SD=0.07).

Univariate report

Firstly, with regard to the type of article published in the journals from the sample (H₂), there is a marked tendency towards the empirical (80.9%) as opposed to the theoretical-essayistic (12.8%) and the methodological (6.3%). Therefore, it is interesting to note both the methodological techniques used in this set of empirical works and the different theories and concepts used in the entirety of the works (Table 1).

If we group the methods into quantitative, qualitative and mixed, we find that the former are present in 46.8% of the papers, the latter in 24.9% and, thirdly, triangulation is present 7.9% of the time (H4). Another important aspect at the methodological level lies in the sample design (RQ₁). In this sense, 61.4% of the papers resort to non-probability sampling compared to 19.4% who use statistical criteria for the handling of representative samples −it should be remembered that the remaining 19.1% are non-empirical studies. In relation to the theoretical framework (H₃), it is worth pointing out that the proportion of texts that rely on some specific conceptual scaffolding as opposed to those that do not is practically the same: 50.2% as opposed to 49.8%. On an individual basis, Media Literacy (8.3%) and Framing Theory (5.1%) are the most widespread.

With regard to the object of study −in its broadest and most generic sense− (H₅), the intermediate link in the process −that is, the message− is the protagonist par excellence (44.6%) of the papers, followed by the audience (21.3%), the source (17.1%) and, lastly, the communication policies and structure (17%). The means or, alternatively, the documentary supports on which the papers focus, as well as their general topics, are distributed as follows (Table 2).

If we aggregate traditional media, we obtain 38.8% of the works compared to 28.2% of digital media (H₁). For their part, the themes around which the studies revolve are also the most diverse (RQ₂), highlighting the uses and motivations of the receivers (11.3%). To complete the percentage review of the variables in the codebook, we find the paradigm to which the articles are ascribed (H6). In this sense, the one that dominates Spanish and Latin American research in communication is positivistic (56%), at a great distance from cultural (21.1%), critical (14.9%) and, most particularly, rhetorical (8%).

Cluster analysis

At the multivariate level, a first two-stage cluster analysis was performed which covers both continuous and categorical variables (Rundle-Thiele & al., 2015), in which we included the 6 most relevant items of the codebook; namely: the theory employed in the articles, the method, the object of the study, the communication medium in which they are centered, the topic in question and the paradigm within which the sample studies are circumscribed.

The silhouette measure of cohesion and separation, which alludes to cluster quality and must exceed the minimum level of 0.0 (Norušis, 2012), is 0.2, an acceptable value although regular. On the other hand, the importance of all the predictive items in the configuration of the groups is very high, since four of them reached the maximum value of 1 (topic, medium, object and method), another one was 0.96 (theory) and the definitive one was 0.78 (paradigm).

Subsequently, the 6 clusters resulting from this first extraction presented a size coefficient of 1.54, which denotes some homogeneity between them, as can be seen in Table 3.

Starting from the creation of the 6 groups, these can be compared according to their SJR impact factor. Analysis of variance (ANOVA) shows statistically significant differences between clusters [F_{6 Clusters x SJR-IF} (5; 1,542)=18.66; p<0.001; ɳ ²=0.057]. More specifically, and after Dunnett's T3 post-hoc test, it was deduced that the “3” and “6” clusters are those that show the greatest imbalances [t(528)=−6.903; p<0.001; d=−0.624], labeled as medium size according to “effect size” (Cohen, 1988; Johnson et al., 2008). Also, the 6 clusters were cross-referenced with the journals where the papers are published, recoding them into two groups: Spanish versus Latin American. The following contingency table reflects the distribution of the two according to publications.

In terms of the values shown in Table 4, significant differences are observed in 4 of the 6 clusters [χ ²(5, N=1,548)=87.52; p<.001; v=0.238]. More specifically, and taking into account the corrected standardized residuals, Latin American journals tend to include article types 3 and 5 to a greater extent, while Spanish journals opt for cases 4 and 6.

The second two-stage cluster analysis differs from the previous one in only one of the items used for its execution: the variable “theory”, which was recoded to take into account only those works that employed some kind of conceptual corpus, considering the rest as “missing”. The silhouette measure of cohesion and separation reached 0.3, an even more acceptable value. As far as the importance of the predictive items is concerned, the method becomes the most prominent element (with a value of 1), followed by the object of study (value 0.92), the theory (value 0.52), the topic (value 0.42) and, ultimately, the paradigm and the medium (both with a value of 0.40). The four clusters derived from the analysis have a size coefficient (from the largest to the smallest) of 2.20, which is not problematic (Tkaczynski, 2017). Below is a profile of the four clusters.

These 4 clusters can also be compared according to their impact factor. Thus, this new analysis of variance reveals the existence of differences between groups according to their SJR-IF once again [F_{4 Clusters x SJR-IF} (3; 773)=26.79; p<0.001; ɳ ²=0.094]. Comparatively, and according to Dunnett's T3 post-hoc test, it is clusters 1 and 4 that reveal the greatest imbalances [t(388)=−7.55; p<0.001; d=−0.831], characterized by an elevated size according to “effect size” (Cohen, 1988; Johnson et al., 2008). Finally, these four clusters were cross-referenced with the journals where the articles are published.

According to the values shown in Table 6, significant differences are observed in all clusters [χ ²(3, N=777)=53.33; p<0.001; v=0.262]. On the basis of the corrected standardized residuals, Latin American journals tend to include article types 1 and 3 to a greater extent, while Spanish journals prefer cases 2 and 4.