Convert 'Year' to a factor variable exceldata$Year <- factor(exceldata$Year) Create a…
- User
Convert 'Year' to a factor variable
excel_dataYear)
Create a formula with fixed effects only
fixed_formula <- as.formula(paste(dependent_var, "~", paste(control_vars, collapse = " + ")))
Fit the logistic regression model with fixed effects using the binomial family
fixed_model <- glm(fixed_formula, family = binomial, data = excel_data)
Extract the grouping factors
grouping_factors <- lme4::lFormula(eval(fixed_formula), data = excel_data)
Create a new data frame with grouping factors added
data_with_grouping <- data.frame(excel_data, grouping_factors$random)
Create a new formula with both fixed and random effects
full_formula <- as.formula(paste(dependent_var, "~", paste(control_vars, collapse = " + "), "+ (1|Year) + (1|CUSIP)"))
Fit the logistic regression model with random effects using the binomial family
full_model <- glmer(full_formula, family = binomial, data = data_with_grouping, control = glmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05)))
Display the summary of the model
summary(full_model)
- Assistant
Your provided code goes through several steps to convert a variable to a factor, create and fit a logistic regression model with fixed effects, and then extends that to a mixed-effect logistic regression model with both fixed and random effects. Below is an explanation of the steps taken in your code:
- First, you're converting a 'Year' variable within the 'excel_data' dataframe to a factor variable, which is common when the year is categorical rather than continuous:
excel_data$Year <- factor(excel_data$Year)- You're constructing a formula for a fixed-effect model by pasting together the dependent variable and control variables, separating each with a plus sign to indicate they are all part of the linear predictors:
fixed_formula <- as.formula(paste(dependent_var, "~", paste(control_vars, collapse = " + ")))- You're fitting a logistic regression model using the
glmfunction, specifyingbinomialas the family since logistic regression is used for binary outcomes:
fixed_model <- glm(fixed_formula, family = binomial, data = excel_data)- You're extracting grouping factors for the random effects using the
lme4::lFormulafunction. This step is typically not necessary for fitting aglm, and it's not clear from your code what the purpose of this step is, but it may be intended as a preliminary step before fitting a mixed model:
grouping_factors <- lme4::lFormula(eval(fixed_formula), data = excel_data)- You're creating a new dataframe that includes the original data as well as any random effects structure identified by
lFormula:
data_with_grouping <- data.frame(excel_data, grouping_factors$random)- You construct a new formula that includes both fixed effects (control variables) and random effects, with random intercepts for 'Year' and 'CUSIP':
full_formula <- as.formula(paste(dependent_var, "~", paste(control_vars, collapse = " + "), "+ (1|Year) + (1|CUSIP)"))- You're fitting a mixed-effect logistic regression model using
glmerfrom thelme4package, specifying the random effects structure:
full_model <- glmer(full_formula, family = binomial, data = data_with_grouping, control = glmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05)))- Finally, you display the summary of the model:
summary(full_model)The
controlargument inglmersuggests that you are using an optimizer called 'bobyqa.' The optionmaxfun = 1e+05sets the maximum number of function evaluations to 100,000, which can be helpful for ensuring the optimizer has enough iterations to converge for complicated models or large datasets.Note that the use of
data_with_groupingappears to be unnecessary for fitting the mixed effects model withglmer. Instead, you could fit the model using the originalexcel_dataand just include random effects of the Year and CUSIP directly in theglmerformula. The originalexcel_datadataset should already include 'Year' and 'CUSIP' if you want to include them as random effects.In summary, your R code snippet demonstrates creating a logistic regression model, followed by converting it into a mixed model with both fixed and random effects. There may be some steps that could be simplified or need adjustment according to the actual structure and content of the data.
- User
rror: No random effects terms specified in formula Traceback:
- lme4::lFormula(eval(fixed_formula), data = excel_data)
- mkReTrms(findbars(RHSForm(formula)), fr)
- stop("No random effects terms specified in formula", call. = FALSE)
- Assistant
The error message "No random effects terms specified in formula" suggests that when you're attempting to use
lme4::lFormula, it's expecting a formula that includes random effects terms, but the formula you have provided (i.e.,fixed_formula) only includes fixed effects.The
lme4::lFormulafunction is typically used to preprocess a mixed-effects model formula that includes random effects. However, as yourfixed_formulaonly represents the fixed effects (since it was created for use in a standardglmlogistic regression model),lme4::lFormuladoesn't find any random effects to process, leading to the error.Since
fixed_formulais intended for aglmmodel and does not include random effects, you should remove or skip this step when you are not fitting a model with random effects.If you want to fit a mixed-effects model using
glmer(and therefore lme4), you should directly specify the formula with the random effects in it and skip thelme4::lFormulafunction.glmercan interpret mixed effects models specified as formulas. You can remove the part of your code that is causing the error:# Removed code that causes the error # grouping_factors <- lme4::lFormula(eval(fixed_formula), data = excel_data) # data_with_grouping <- data.frame(excel_data, grouping_factors$random)Instead, directly use the
full_formulathat includes both fixed and random effects to fit the mixed-effects model:full_formula <- as.formula(paste(dependent_var, "~", paste(control_vars, collapse = " + "), "+ (1|Year) + (1|CUSIP)")) full_model <- glmer(full_formula, family = binomial, data = excel_data, control = glmerControl(optimizer = "bobyqa", optCtrl = list(maxfun = 1e+05))) summary(full_model)This should resolve the error because
glmerexpects a formula that includes random effects, whichfull_formuladoes. Note that in this fixed-up code, we useexcel_datadirectly rather than creatingdata_with_grouping, which is unnecessary in this context.If you happen to have additional random effects that you wish to include, or you need to do preprocessing of your data to include variables that represent random effects, you would adjust the formula accordingly to include those terms. However, the key here is that
lme4::lFormulais not necessary for standard use ofglmer. - User
Warning message: “Unknown or uninitialised column:
EQT_MANUFACTURING.” Error in$<-: ! Assigned datafactor(excel_data$EQT_MANUFACTURING)must be compatible with existing data. ✖ Existing data has 2075 rows. ✖ Assigned data has 0 rows. ℹ Only vectors of size 1 are recycled. Caused by error invectbl_recycle_rhs_rows(): ! Can't recycle input of size 0 to size 2075. Traceback:$<-(*tmp*, EQT_MANUFACTURING, value = structure(integer(0), levels = character(0), class = "factor"))$<-.tbl_df(*tmp*, EQT_MANUFACTURING, value = structure(integer(0), levels = character(0), class = "factor"))- tbl_subassign(x, i = NULL, as_string(name), list(value), i_arg = NULL, . j_arg = name, value_arg = substitute(value))
- vectbl_recycle_rhs_rows(value, fast_nrow(xo), i_arg = NULL, value_arg, . call)
- withCallingHandlers(for (j in seq_along(value)) { . if (!is.null(value[[j]])) { . value[[j]] <- vec_recycle(value[[j]], nrow) . } . }, vctrs_error_recycle_incompatible_size = function(cnd) { . abort_assign_incompatible_size(nrow, value, j, i_arg, value_arg, . cnd, call = call) . }, vctrs_error_scalar_type = function(cnd) { . abort_assign_vector(value, j, value_arg, cnd, call = call) . })
- vec_recycle(value[[j]], nrow)
- stop_recycle_incompatible_size(x_size = 0L, size = 2075L, x_arg = "", . call = )
- stop_vctrs(x_size = x_size, y_size = size, x_arg = x_arg, class = c("vctrs_error_incompatible_size", . "vctrs_error_recycle_incompatible_size"), call = call)
- abort(message, class = c(class, "vctrs_error"), ..., call = call)
- signal_abort(cnd, .file)
- signalCondition(cnd)
- (function (cnd)
. {
. abort_assign_incompatible_size(nrow, value, j, i_arg, value_arg,
. cnd, call = call)
. })(structure(list(message = "", trace = structure(list(call = list(
. IRkernel::main(), kernelexecute(msg),
. tryCatch(evaluate(requestcode, envir = .GlobalEnv,
. output_handler = oh, stop_on_error = 1L), interrupt = function(cond) {
. log_debug("Interrupt during execution")
. interrupted <<- TRUE
. }, error = .selfcontentsrc[[i]], envir = envir, enclos = enclos,
. debug = debug, last = i == length(out), use_try = stop_on_error !=
. 2L, keep_warning = keep_warning, keep_message = keep_message,
. log_echo = log_echo, log_warning = log_warning, output_handler = output_handler,
. include_timing = include_timing), timing_fn(handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,
. envir, enclos, user_handlers)), warning = wHandler, error = eHandler,
. message = mHandler))), handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,
. envir, enclos, user_handlers)), warning = wHandler, error = eHandler,
. message = mHandler)), try(f, silent = TRUE), tryCatch(expr,
. error = function(e) {
. call <- conditionCall(e)
. if (!is.null(call)) {
. if (identical(call[[1L]], quote(doTryCatch)))
. call <- sys.call(-4L)
. dcall <- deparse(call, nlines = 1L)
. prefix <- paste("Error in", dcall, ": ")
. LONG <- 75L
. sm <- strsplit(conditionMessage(e), "\n")[[1L]]
. w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L],
. type = "w")
. if (is.na(w))
. w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],
. type = "b")
. if (w > LONG)
. prefix <- paste0(prefix, "\n ")
. }
. else prefix <- "Error : "
. msg <- paste0(prefix, conditionMessage(e), "\n")
. .Internal(seterrmessage(msg[1L]))
. if (!silent && isTRUE(getOption("show.error.messages"))) {
. cat(msg, file = outFile)
. .Internal(printDeferredWarnings())
. }
. invisible(structure(msg, class = "try-error", condition = e))
. }), tryCatchList(expr, classes, parentenv, handlers),
. tryCatchOne(expr, names, parentenv, handlers[[1L]]), doTryCatch(return(expr),
. name, parentenv, handler), withCallingHandlers(withVisible(eval_with_user_handlers(expr,
. envir, enclos, user_handlers)), warning = wHandler, error = eHandler,
. message = mHandler), withVisible(eval_with_user_handlers(expr,
. envir, enclos, user_handlers)), eval_with_user_handlers(expr,
. envir, enclos, user_handlers), eval(expr, envir, enclos),
. eval(expr, envir, enclos),
$<-(*tmp*, EQT_MANUFACTURING, . value =<fct>),$<-.tbl_df(*tmp*, EQT_MANUFACTURING, . value =<fct>), tbl_subassign(x, i = NULL, as_string(name), . list(value), i_arg = NULL, j_arg = name, value_arg = substitute(value)), . vectbl_recycle_rhs_rows(value, fast_nrow(xo), i_arg = NULL, . value_arg, call), withCallingHandlers(for (j in seq_along(value)) { . if (!is.null(value[[j]])) { . value[[j]] <- vec_recycle(value[[j]], nrow) . } . }, vctrs_error_recycle_incompatible_size = function(cnd) { . abort_assign_incompatible_size(nrow, value, j, i_arg, . value_arg, cnd, call = call) . }, vctrs_error_scalar_type = function(cnd) { . abort_assign_vector(value, j, value_arg, cnd, call = call) . }), vec_recycle(value[[j]], nrow), stop_recycle_incompatible_size(x_size = 0L, . size = 2075L, x_arg = "", call =<env>), stop_vctrs(x_size = x_size, . y_size = size, x_arg = x_arg, class = c("vctrs_error_incompatible_size", . "vctrs_error_recycle_incompatible_size"), call = call), . abort(message, class = c(class, "vctrs_error"), ..., call = call)), . parent = c(0L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 6L, 9L, 10L, 4L, . 12L, 13L, 13L, 15L, 16L, 17L, 18L, 19L, 13L, 13L, 13L, 23L, . 24L, 0L, 0L, 27L, 28L, 29L, 29L, 0L, 32L, 33L), visible = c(TRUE, . TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, . TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, . TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, FALSE, FALSE, . FALSE, FALSE, FALSE), namespace = c("IRkernel", NA, "IRkernel", . NA, "base", "base", "base", "base", "base", "base", "base", . "evaluate", "evaluate", "evaluate", "evaluate", "base", "base", . "base", "base", "base", "base", "base", "evaluate", "base", . "base", "base", "tibble", "tibble", "tibble", "base", "vctrs", . "vctrs", "vctrs", "rlang"), scope = c("::", NA, "local", . NA, "::", "local", "local", "local", "local", "local", "local", . "::", ":::", "local", "local", "::", "::", "local", "local", . "local", "::", "::", ":::", "::", "::", "::", ":::", ":::", . ":::", "::", "::", ":::", ":::", "::"), error_frame = c(FALSE, . FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, . FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, . FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, . TRUE, FALSE, FALSE, FALSE, FALSE, FALSE)), row.names = c(NA, . -34L), version = 2L, class = c("rlang_trace", "rlib_trace", "tbl", . "data.frame")), parent = NULL, x_size = 0L, y_size = 2075L, x_arg = "", . rlang = list(inherit = TRUE), call = vectbl_recycle_rhs_rows(value, . fast_nrow(xo), i_arg = NULL, value_arg, call)), class = c("vctrs_error_incompatible_size", . "vctrs_error_recycle_incompatible_size", "vctrs_error", "rlang_error", . "error", "condition"))) - abort_assign_incompatible_size(nrow, value, j, i_arg, value_arg, . cnd, call = call)
- tibble_abort(bullets(paste0("Assigned data ", tick(as_label(value_arg)),
. " must be compatible with ", target, ":"), x = existing,
. x = new, i = if (nrow != 1) "Only vectors of size 1 are recycled",
. i = if (nrow == 1 && vec_size(value[[j]]) != 1) "Row updates require a list value. Do you need
list()oras.list()?"), . expected = nrow, actual = vec_size(value[[j]]), j = j, parent = parent, . call = call) - abort(x, class, ..., call = call, parent = parent, use_cli_format = TRUE)
- signal_abort(cnd, .file)
- Assistant
The error and warning messages you're encountering suggest that you are trying to assign a new value to a column in your
excel_datadataframe, specifically a column namedEQT_MANUFACTURING, but there is an issue with the data being assigned.Here's what the error message indicates:
- The warning "Unknown or uninitialised column:
EQT_MANUFACTURING." indicates thatEQT_MANUFACTURINGdoes not exist in yourexcel_datadataframe or it has not been initialized properly. - The error message that follows is complaining about the size of the data you're trying to assign to the
EQT_MANUFACTURINGcolumn. The existingexcel_datahas 2075 rows, but the data you're trying to assign has 0 rows.
This kind of situation can happen if there's a typo in the column name or if the column is expected to be created upon assignment, but the right side of the assignment, i.e.
factor(excel_data$EQT_MANUFACTURING), evaluates to an empty vector or perhaps the variableexcel_data$EQT_MANUFACTURINGis entirely missing from your dataset.Let's assume you wanted to convert the column
EQT_MANUFACTURINGinexcel_datato a factor. Here is a code snippet that would do that correctly (but only if the column already exists inexcel_data):excel_data$EQT_MANUFACTURING <- factor(excel_data$EQT_MANUFACTURING)To diagnose the issue, let's take a step-by-step approach:
- Make sure
excel_datais the correct dataframe and it is properly loaded into the R environment. - Confirm that
EQT_MANUFACTURINGexists within that dataframe. You can check the column names withnames(excel_data)to see if it is listed. - If the column does not exist, you'll need to figure out why it's missing and correct that issue first.
- If the column does exist but for some reason has no data, you need to investigate why it is empty before you convert it to a factor.
To make sure you do not try to convert a non-existing column to a factor, you might want to add a conditional check:
if("EQT_MANUFACTURING" %in% names(excel_data)) { excel_data$EQT_MANUFACTURING <- factor(excel_data$EQT_MANUFACTURING) } else { warning("Column 'EQT_MANUFACTURING' does not exist in 'excel_data' dataframe.") }This conditional will prevent the attempt to manipulate a column that does not exist, and instead give you a warning to alert you to the problem. This can help prevent the error by ensuring the code does not run if the conditions are not right.
- The warning "Unknown or uninitialised column: