Pre-trained Concreteness Detection Model for Advice

Description

This model was pre-trained on 3289 examples of feedback on different tasks (e.g. writing a cover letter, boggle, workplace annual reviews). All of those documents were annotated by research assistants for concreteness, and this model simulates those annotations on new documents.

Model pre-trained on advice data.

Usage

adviceModel

adviceModel(texts, num.mc.cores = 1)

Arguments

Format

A pre-trained glmnet model

Value

numeric Vector of concreteness ratings.

Source

Yeomans (2020). A Concrete Application of Open Science for Natural Language Processing.

Pre-trained advice concreteness features

Description

For internal use only. This dataset demonstrates the ngram features that are used for the pre-trained adviceModel.

Usage

adviceNgrams

Format

A (truncated) matrix of ngram feature counts for alignment to the pre-trained advice glmnet model.

Source

Yeomans (2020). A Concrete Application of Open Science for Natural Language Processing.

Concreteness mTurk Word List

Description

Word list from Paetzold & Specia (2016). A list of 85,942 words where concreteness was imputed using word embeddings.

Usage

bootstrap_list

Format

A data frame with 85,942 rows and 2 variables.

Word

character text of a word with an entry in this dictionary

Conc.M

predicted concreteness score for that word (from 100-700)

Source

#' Paetzold, G., & Specia, L. (2016, June). Inferring psycholinguistic properties of words. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 435-440).

Cleaning weird encodings

Description

Handles curly quotes, umlauts, etc.

Usage

cleanpunct(text)

Arguments

Value

character Vector of clean strings.

Text Cleaner

Description

background function to load.

Usage

cleantext(
  text,
  language = "english",
  punct = FALSE,
  stop.words = TRUE,
  number.words = TRUE
)

Arguments

Value

character Vector of cleaned strings.

Open-Domain Concreteness Dictionaries

Description

background function to load

Usage

concDict(
  texts,
  wordlist = NULL,
  stop.words = TRUE,
  number.words = TRUE,
  shrink = FALSE,
  fill = FALSE,
  minwords = 0,
  num.mc.cores = 1
)

Arguments

Value

concreteness score for each document

Contraction Expander

Description

background function to load.

Usage

ctxpand(text)

Arguments

Value

character Vector of sentences without contractions.

Concreteness Scores

Description

Detects linguistic markers of concreteness in natural language. This function is the workhorse of the doc2concrete package, taking a vector of text documents and returning an equal-length vector of concreteness scores.

Usage

doc2concrete(
  texts,
  domain = c("open", "advice", "plans"),
  wordlist = doc2concrete::mturk_list,
  stop.words = TRUE,
  number.words = TRUE,
  shrink = FALSE,
  fill = FALSE,
  uk_english = FALSE,
  num.mc.cores = 1
)

Arguments

Details

In principle, concreteness could be measured from any english text. However, the definition and interpretation of concreteness may vary based on the domain. Here, we provide a domain-specific pre-trained classifier for concreteness in advice & feedback data, which we have empirically confirmed to be robust across a variety of contexts within that domain (Yeomans, 2021).

The training data for the advice classifier includes both second-person (e.g. "You should") and third-person (e.g. "She should") framing, including some names (e.g. "Riley should"). For consistency, we anonymised all our training data to replace any names with "Riley". If you are working with a dataset that includes the names of advice recipients, we recommend you convert all those names to "Riley" as well, to ensure optimal performance of the algorithm (and to respect their privacy).

There are many domains where such pre-training is not yet possible. Accordingly, we provide support for two off-the-shelf concreteness "dictionaries" - i.e. document-level aggregations of word-level scores. We found that that have modest (but consistent) accuracy across domains and contexts. However, we still encourage researchers to train a model of concreteness in their own domain, if possible.

Value

A vector of concreteness scores, with one value for every item in 'text'.

References

Yeomans, M. (2021). A Concrete Application of Open Science for Natural Language Processing. Organizational Behavior and Human Decision Processes, 162, 81-94.

Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904-911.

Paetzold, G., & Specia, L. (2016, June). Inferring psycholinguistic properties of words. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 435-440).

Examples

data("feedback_dat")

doc2concrete(feedback_dat$feedback, domain="open")


cor(doc2concrete(feedback_dat$feedback, domain="open"),feedback_dat$concrete)

Doublestacker

Description

background function to load

Usage

doublestacker(wdcts)

Arguments

Value

Token count matrix with no doubled column names.

Personal Feedback Dataset

Description

A dataset containing responses from people on Mechanical Turk, writing feedback to a recent collaborator, that were then scored by other Turkers for feedback specificity. Note that all proper names of advice recipients have been substituted with "Riley" - we recommend the same in your data.

Usage

feedback_dat

Format

A data frame with 171 rows and 2 variables:

feedback

character text of feedback from writers

concrete

numeric average specificity score from readers

Source

Blunden, H., Green, P., & Gino, F. (2018).

"The Impersonal Touch: Improving Feedback-Giving with Interpersonal Distance."

Academy of Management Proceedings, 2018.

Concreteness mTurk Word List

Description

Word list from Brysbaert, Warriner & Kuperman (2014). A list of 39,954 words that have been hand-annotated by crowdsourced workers for concreteness.

Usage

mturk_list

Format

A data frame with 39,954 rows and 2 variables.

Word

character text of a word with an entry in this dictionary

Conc.M

average concreteness score for that word (from 1-5)

Source

Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46(3), 904-911.

Ngram Tokenizer

Description

Tally bag-of-words ngram features

Usage

ngramTokens(
  texts,
  wstem = "all",
  ngrams = 1,
  language = "english",
  punct = TRUE,
  stop.words = TRUE,
  number.words = TRUE,
  per.100 = FALSE,
  overlap = 1,
  sparse = 0.995,
  verbose = FALSE,
  vocabmatch = NULL,
  num.mc.cores = 1
)

Arguments

Details

This function produces ngram featurizations of text based on the quanteda package. This provides a complement to the doc2concrete function by demonstrating How to build a feature set for training a new detection algorithm in other contexts.

Value

a matrix of feature counts

Examples

dim(ngramTokens(feedback_dat$feedback, ngrams=1))
dim(ngramTokens(feedback_dat$feedback, ngrams=1:3))

Overlap cleaner

Description

background function to load

Usage

overlaps(high, low, cutoff = 1, verbose = FALSE)

Arguments

Value

Combined token count matrix.

Pre-trained Concreteness Detection Model for Plan-Making

Description

This model was pre-trained on 5,172 examples of pre-course plans from online courses at HarvardX. Each plan was annotated by research assistants for concreteness, and this model simulates those annotations on new plans.

Model pre-trained on planning data.

Usage

planModel

planModel(texts, num.mc.cores = 1)

Arguments

Format

A pre-trained glmnet model

Value

numeric Vector of concreteness ratings.

Source

Yeomans (2020). A Concrete Application of Open Science for Natural Language Processing.

Pre-trained plan concreteness features

Description

For internal use only. This dataset demonstrates the ngram features that are used for the pre-trained planModel.

Usage

planNgrams

Format

A (truncated) matrix of ngram feature counts for alignment to the pre-trained planning glmnet model.

Source

Yeomans (2020). A Concrete Application of Open Science for Natural Language Processing.

Conditional Stemmer

Description

background function to load

Usage

stemexcept(sentence, excepts, language = "english")

Arguments

Value

Sentence of stemmed words.

Stemmer

Description

background function to load.

Usage

stemmer(text, wstem = "all", language = "english")

Arguments

Value

Sentence of stemmed words.

Text Formatter

Description

background function to load.

Usage

textformat(text, punct = FALSE)

Arguments

Value

character Vector of cleaned strings.

UK to US Conversion dictionary

Description

For internal use only. This dataset contains a quanteda dictionary for converting UK words to US words. The models in this package were all trained on US English.

Usage

uk2us

Format

A quanteda dictionary with named entries. Names are the US version, and entries are the UK version.

Source

Borrowed from the quanteda.dictionaries package on github (from user kbenoit)

UK to US conversion

Description

background function to load.

Usage

usWords(text)

Arguments

Value

character Vector of Americanized strings.

Feature Count Matcher

Description

background function to load

Usage

vocabmatcher(hole, peg)

Arguments

Value

Token counts matrix from new data, with column names that match the model data.