---
title: "The extended xpose data object"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{The extended xpose data object}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.align='center'
)
```

```{r setup, echo=FALSE, include = FALSE}
library(dplyr)
library(xpose)
library(xpose.xtras)
```

## How to get the object

The `xpose_data` object (aka: `xpdb`) gets a few extensions in this package. Many of these are added to accommodate new plots and features, but in most cases these features are cross-compatible with a typical `xpose_data` object as well. The extended version of `xpose_data` has the class `xp_xtra`, and is also referred to as `xpdb_x`.

Conversion of an `xpose_data` object is simple.

```{r conversion}
xpdb_converted <- xpdb_ex_pk %>%
  as_xpdb_x()

# To verify:
xpdb_ex_pk
xpdb_converted
```

As shown, there are some minor changes to the output for `print()` to help confirm if an object is the extended version. However, this is implemented merely as an S3 class, so all `xpose` functions will continue to work with this new object; the current package is also written in a way to work even if the `xp_xtra` object loses that class.

Notably, there is no convenience function to read a model output directly to an `xp_xtra` object, so all objects labelled as `xp_xtra` start as `xpose_data`.

Many example `xp_xtra` objects are included in this package, covering a variety of special cases. 

## What features are available?

### Tidying up

It is by design that `xpose` attempts to align well with the `tidyverse` family of R packages, so one focus of the extension package is to make that alignment a bit more consistent. For example, `xpose::set_var_types` accepts character vectors of column names, but not tidyselection. As such, columns named in a consistent and tidyselect-friendly way cannot be used to an advantage. A minimal example can be seen below, but of course there are more complex situations where this is convenient.

```{r set_var_types}
# Unset all example covariates
xpdb_ex_covs <- xp_var(xpdb_ex_pk, type = c("catcov","contcov"), .problem=1) %>% 
  pull(col)
xpdb_ex_covs
no_covs <- set_var_types(xpdb_ex_pk, .problem=1, na = xpdb_ex_covs)

# set_var_types on xpose_data objects uses xpose::set_var_types
set_var_types(no_covs, .problem=1, catcov = c("MED1","MED2")) %>%
  xp_var(type = c("catcov","contcov"), .problem=1) %>% 
  pull(col)
no_covs %>%
  as_xpdb_x() %>%
  set_var_types(catcov = starts_with("MED"), .problem=1) %>%
  xp_var(type = c("catcov","contcov"), .problem=1) %>% 
  pull(col)
```

Tidyselection is used fairly heavily in the `xpose.xtras` package where it seems intuitive to include it. There are likely functions that could use it but do not currently. As such, the documentation does attempt to distinguish where `tidyselect` is expected.

### Labels, units and (also) levels

Another feature includes a few visual confirmations of variable units, labels and (new to this package) levels. In `xpose`, users are able to add units and labels to any variable, which are stored in the object and are theoretically used somewhere. This package attempts to use these communication features more liberally in plots and for the user, but broader implementation (some of which require changes to existing `xpose` functions) is not complete just yet.

With labels and units, these are applied using the conventional `xpose` functions. However, unlike in `xpose`, these can be confirmed using `list_vars()` in `xpose.xtras`.

```{r set_labels_units}
w_unit_labs <- xpdb_x %>%
  set_var_labels(AGE="Age", MED1 = "Digoxin", .problem = 1) %>%
  set_var_units(AGE="yrs")
list_vars(w_unit_labs, .problem = 1)
```

Levels can also be added to any variable, but can be especially useful for categorical covariates (`catcov`) and categorical DVs (type `catdv`, added by this package). The documentation for `set_var_levels()` can be referenced for more information about this, but the example below shows how this feature can be used and checked.

```{r set_levels}
w_levels <- w_unit_labs  %>%
  set_var_levels(SEX=lvl_sex(), MED1 = lvl_bin())
list_vars(w_levels, .problem = 1)
```

Labels, units and levels also appear in new plotting functions. There are some more complex functions where this renaming has not been implemented, but the plan is to have it be universal eventually.

```{r plot_cont, fig.width=unit(6,"in"), fig.height=unit(3,"in")}
eta_vs_contcov(w_unit_labs,etavar=ETA1, quiet=TRUE)
```

```{r plot_cat, fig.width=unit(6,"in"), fig.height=unit(3,"in")}
eta_vs_catcov(w_levels,etavar=ETA1, quiet=TRUE)
```

Note in the categorical plots, the number of individuals represented by each category is underneath the value. This is similar to the presentation in the package [`pmplots`](https://metrumresearchgroup.github.io/pmplots/), but uses the existing `xpose` framework. This option requires the use of an `xp_xtra` object, and can be disabled with the argument `show_n=FALSE`.

### Parameter tables

The `get_prm()` function in `xpose` has been extended to output coefficient of variation percent (CV%) for $\omega^2$ parameters and shrinkages where relevant.

```{r get_prm}
get_prm(pheno_final) %>%
  select(-c(fixed,m,n))
```

There are a few options to change CV% where an opinion might conflict with the default. One such option is to add a description of how the parameter is used, which results in an exact CV% determined through integration to be used. For example, if variability on the weight-normalized clearance and volume of distribution parameters in one of the example models was logit-normally distributed, it could be described as follows.

```{r get_prm2}
pheno_final %>%
   add_prm_association(CLpkg~logit(IIVCL),Vpkg~logit(IIVV)) %>%
   get_prm() %>%
  select(-c(fixed,m,n))
```

There is a fair amount of complexity in the `get_prm()` extensions, including ability to change parameter value while also changing standard error and relevant variance-covariance (in position for the `.cov` and `.cor` files). See `?add_prm_association` and `?mutate_prm`.

## Cross-compatibility

In various warning messages, the `xp_xtras` object is referred to as "cross-compatible". This word choice is intended to highlight that many functions developed for `xp_xtras` objects will still work (perhaps with some diminished functionality) for `xpose_data` objects, and that the inverse is of course true.

Another way this cross-compatibility is maintained is via custom themes. There may be `xpose` users concerned about adopting `xpose.xtras` because with the new plot elements they would have to declare new aesthetic defaults. Conveniently, the `xp_xtra` theme is derived in a way such that aesthetics for defined plot elements are re-used for similar new elements, meaning there is less (if any) updates needed to use a custom theme beyond using something like the following.

```{r xp_themes, fig.width=unit(6,"in"), fig.height=unit(3,"in")}
favorite_theme <- xpose::theme_xp_xpose4() # stand-in for "custom" theme

eta_vs_catcov(w_levels,etavar=ETA1, quiet=TRUE)
eta_vs_catcov(w_levels,etavar=ETA1, quiet=TRUE, xp_theme = favorite_theme)

```

Note that worked even though `boxplot_fill` is not defined in the `xpose::theme_xp_xpose4()`. To update an existing theme object to one for the extension package (instead of using the old theme in the `xp_theme` argument), simply use `updated_theme <- xp_xtra_theme(old_theme)`.

## Convenience

`xpose` has a collection of getter and setter functions to interact with parts of an `xpose_data` object, but these tend to be lower level and not exported. To add this convenience and make it user-friendly, a few functions have been added.

Properties from a model summary can now be pulled without using `xpose::get_summary()`.

```{r get_props}
pheno_final %>% get_shk()
pheno_final %>% get_shk("eps")
pheno_final %>% get_prop("ofv")
pheno_final %>% get_prop("descr")
```

`xpose` has certain expectations around how model descriptions are included in model code. At time of writing, it is expected the format is something like "; 2. Description: (text)". To add some flexibility, users can either use `set_prop()` to change description directly, or pull it from a more generic scan of the model code comments using `desc_from_comments()`.

```{r describer}
pheno_final %>% desc_from_comments() %>% get_prop("descr")
```

It can be helpful to add information to the model data that is available elsewhere in the `xpose_data` object, such as individual objective functions. Currently, those can be added using `backfill_iofv()`, which is most often used in the context of `xpose_set` objects. Theoretically more backfill functions can be made available.