Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
533 changes: 270 additions & 263 deletions R/sql_server.R

Large diffs are not rendered by default.

34 changes: 17 additions & 17 deletions R/utils.R
Original file line number Diff line number Diff line change
Expand Up @@ -19,51 +19,51 @@
#' @export

get_env_var <- function(var_name) {

# if running on server, use Sys.getenv()
if (Sys.getenv()["USERNAME"][[1]] == "rstudio-connect") {

return(Sys.getenv(var_name))

# if running locally, try keyring first
} else {

# try keyring
kr <- try({
var <- keyring::key_get(var_name)
TRUE },
silent = TRUE)
kr <- try(
{
var <- keyring::key_get(var_name)
TRUE
},
silent = TRUE
)

# if keyring worked, return
if (kr[1] == TRUE) {
return(var)

# otherwise, try Sys.getenv
} else {

sys_env <- try({
var <- Sys.getenv(var_name)
TRUE},
silent = TRUE)
sys_env <- try(
{
var <- Sys.getenv(var_name)
TRUE
},
silent = TRUE
)

# if Sys.gentenv worked, give warning and return
if (sys_env[1] == TRUE & var != "") {

warning(glue(
warning(glue::glue(
"variable {var_name} found in Renviron file but not windows credentials. ",
"Update your process to use keyring and not .Renviron file by doing the ",
"following: \n",
" 1. Run: keyring::key_set(service = '{var_name}') \n",
" 2. Enter the variable value in the text box \n",
" 3. Delete the variable {var_name} from your .Renviron file \n"
)
)
))

return(var)

# otherwise, variable not found
} else {
stop(glue("variable {var_name} not found using keyring or Sys.getenv()"))
stop(glue::glue("variable {var_name} not found using keyring or Sys.getenv()"))
}
}
}
Expand Down
4 changes: 2 additions & 2 deletions R/zzz.R
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
#.onAttach <- function(libname, pkgname) {
# .onAttach <- function(libname, pkgname) {
# packageStartupMessage("WARNING: This package is still in development, use at own risk.")
#}
# }
125 changes: 64 additions & 61 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ knitr::opts_chunk$set(
comment = "#>",
fig.path = "tools/README-"
)

```

# SQL R Tools
Expand Down Expand Up @@ -69,37 +68,35 @@ again*
To set up GitHub PAT in the Git Credential Store & install package:

```{r, install, eval=FALSE}

# Install required packages
install.packages("usethis")
install.packages("gitcreds")
install.packages("remotes")
install.packages("pak")

# set up GitHub PAT. Note: if a GitHub PAT has already been saved, the following
# will give an option to replace the credentials. Run the function, select the
# relevant option and update the PAT with the one just copied from GitHub.
# will give an option to replace the credentials. Run the function, select the
# relevant option and update the PAT with the one just copied from GitHub.
gitcreds::gitcreds_set()

# Check has been set properly, the below should return a list with details of:
# protocol, host, username and password. None of these should be NA, the
# password should have "<-- hidden -->".
# protocol, host, username and password. None of these should be NA, the
# password should have "<-- hidden -->".
gitcreds::gitcreds_get()

# If the above doesn't look correct:
# 1. Make sure you don't have GITHUB_PAT set in your Renviron file. If you do,
# 1. Make sure you don't have GITHUB_PAT set in your Renviron file. If you do,
# delete it, restart R, and check it's no longer in use by running
# Sys.getenv("GITHUB_PAT"), this should return "".
# 2. Delete your GITHUB_PAT on GitHub at https://github.com/settings/tokens and
# re-create the PAT using the instructions above (this may have failed if
# you already had a GitHub PAT set up).
# Sys.getenv("GITHUB_PAT"), this should return "".
# 2. Delete your GITHUB_PAT on GitHub at https://github.com/settings/tokens and
# re-create the PAT using the instructions above (this may have failed if
# you already had a GitHub PAT set up).

# If the output of gitcreds::gitcreds_get() still looks incorrect, contact the
# author of this package.
# If the output of gitcreds::gitcreds_get() still looks incorrect, contact the
# author of this package.

# install the package (see notes below if this fails)
pak::pkg_install("Notts-HC/SQLRtools")

```

If you have a `GITHUB_PAT` set up in the Renviron file **delete it**. The
Expand Down Expand Up @@ -128,8 +125,6 @@ Sys.setenv(GITHUB_PAT = gitcreds::gitcreds_get(use_cache = FALSE)$password)

# install using remotes
remotes::install_github("Notts-HC/SQLRtools")


```

## About
Expand All @@ -154,7 +149,6 @@ unloadNamespace("SQLRtools")

```{r, error = TRUE}
covr::package_coverage()

```

## Using the package
Expand All @@ -167,55 +161,57 @@ server.
The below gives examples of connecting to a MSSQL server and MYSQL server:

```{r sql connection, eval=FALSE}

library(SQLRtools)


# set connection to MS SQL server
ms_sql_server <- sql_server$new(driver = "SQL Server",
server = get_env_var("MSSQL_SERVER"),
database = get_env_var("MSSQL_DATABASE"))
ms_sql_server <- sql_server$new(
driver = "SQL Server",
server = get_env_var("MSSQL_SERVER"),
database = get_env_var("MSSQL_DATABASE")
)

# set connect to MySQL server
my_sql_server <- sql_server$new(driver = "MySQL ODBC 8.0 Unicode Driver",
server = get_env_var("HOST_NAME"),
database = get_env_var("MYSQL_DB"),
port = get_env_var("MYSQL_PORT"),
uid = get_env_var("MYSQL_USER"),
pwd = get_env_var("MYSQL_PASSWORD"))

my_sql_server <- sql_server$new(
driver = "MySQL ODBC 8.0 Unicode Driver",
server = get_env_var("HOST_NAME"),
database = get_env_var("MYSQL_DB"),
port = get_env_var("MYSQL_PORT"),
uid = get_env_var("MYSQL_USER"),
pwd = get_env_var("MYSQL_PASSWORD")
)
```

The methods listed above can now be used with these connections to:

##### Upload & query data

```{r basic sql examples, eval=FALSE}

# create a basic temp table
my_data <- data.frame(a = c("a", "b", "c"),
b = 1:3)
my_data <- data.frame(
a = c("a", "b", "c"),
b = 1:3
)

# upload as a temporary table - note that generally close_conn should be TRUE
# (which is the default setting), but it needs to be FALSE here so the
# (which is the default setting), but it needs to be FALSE here so the
# connection isn't shut after uploading the temporary table, as this would drop
# the temporary table straight away.
ms_sql_server$upload(data = my_data,
table_name = "#SQLRtools_example",
close_conn = FALSE)
# the temporary table straight away.
ms_sql_server$upload(
data = my_data,
table_name = "#SQLRtools_example",
close_conn = FALSE
)

# get the data
sql_data <- ms_sql_server$get("SELECT *
sql_data <- ms_sql_server$get("SELECT *
FROM #SQLRtools_example")


```


##### Explore databases & their objects

```{r details of objects on server, eval=FALSE}

# get databases in server
ms_sql_dbs <- ms_sql_server$databases()

Expand All @@ -226,50 +222,57 @@ my_sql_db_tables <- ms_sql_server$db_tables(database = ms_sql_dbs$name[20])
my_sql_db_views <- ms_sql_server$db_views(database = ms_sql_dbs$name[20])

# get meta data of table in given tables
my_sql_meta_data <- ms_sql_server$meta_data(database = ms_sql_dbs$name[20],
objects = my_sql_db_views$view_name[1:5],
details = FALSE)
my_sql_meta_data <- ms_sql_server$meta_data(
database = ms_sql_dbs$name[20],
objects = my_sql_db_views$view_name[1:5],
details = FALSE
)

names(my_sql_meta_data)[1]
View(my_sql_meta_data[1][[1]])

```

##### Avoid the "Invalid Descriptor Index" issue

```{r invalid descriptor index example, eval=FALSE}

# create a basic temp table
my_data <- data.frame(a = c("a", "b", "c"),
b = 1:3)
my_data <- data.frame(
a = c("a", "b", "c"),
b = 1:3
)

# upload as a temporary table
ms_sql_server$upload(data = my_data,
table_name = "#SQLRtools_example",
close_conn = FALSE)
# upload as a temporary table
ms_sql_server$upload(
data = my_data,
table_name = "#SQLRtools_example",
close_conn = FALSE
)

# change the data type of column a to varchar(max)
ms_sql_server$run("ALTER TABLE #SQLRtools_example
ALTER COLUMN a varchar(max);",
close_conn = FALSE)
close_conn = FALSE
)

# try extracting the data
ms_sql_server$get("SELECT * FROM #SQLRtools_example",
close_conn = FALSE)
close_conn = FALSE
)

# use method order_object_fields to avoid issue

# get all the fields ordered by data type
table_fields <- ms_sql_server$order_object_fields(object = "#SQLRtools_example",
close_conn = FALSE)
table_fields <- ms_sql_server$order_object_fields(
object = "#SQLRtools_example",
close_conn = FALSE
)

# now extract all the data
ms_sql_server$get(glue("SELECT {table_fields}
ms_sql_server$get(glue("SELECT {table_fields}
FROM #SQLRtools_example"))

# note that the above table only has 2 fields. Where your trying to run
# select * on a table with a lot of fields and multiple varchar(max) or
# geometry feilds, the above makes life a lot easier (notwithstanding the fact
# it's good practice to avoid 'select *' in SQL where possible).

# note that the above table only has 2 fields. Where your trying to run
# select * on a table with a lot of fields and multiple varchar(max) or
# geometry feilds, the above makes life a lot easier (notwithstanding the fact
# it's good practice to avoid 'select *' in SQL where possible).
```
1 change: 1 addition & 0 deletions SQLRtools.Rproj
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
Version: 1.0
ProjectId: db9d44e7-f57d-4157-a5ba-a48925b20d99

RestoreWorkspace: Default
SaveWorkspace: Default
Expand Down
Loading