Speed up microdata anonymization process

#Introduction

One of the main use of the agrisvy package is to facilitate files and data management during anonymization of dataset, spacially from household survey including agricultural survey

the package allow to create all the folders you need, it generate initial,pre-populated, R scripts you need for deta pre-processing and anonymization. It also generate an initial sample report in R markdown format and contains function that appends the report during the anonymizaion process.

This article shows howe to setup the anonymization files, folders, scripts and reports.

library(agrisvyr)

Agrisvy object

agrisvy <- agrisvyr::createAgrisvy(
  svyName    = "AGRIS survey 2023",
  author     = "AgriSvey Team",
  language   = "en",
  workingDir = "C:/Users/USER/Documents/agrisvyr/anonymization",
  dataDir    = "C:/Users/USER/Documents/agrisvyr/example data",
  type       = ".dta"
)

print(agrisvy)

Creating working folders, files,scripts and report

there two main inputs here: the path to the folder wher the microdata are located and the working directory for the aanonymization. All the working folders and files will be created in the working directory. However,the microdata can be in a different location. There are other input such as the names of the folders (if we want to use another language),the data type.

In order to facilitate the initial settings, we should create an agrisvy object to store all this information once for all and then apply the function setup_anonymization() to the object to create the folder and files.

# agrisvyr::setup_anonymization(agrisvy,overwrite = TRUE)

Now let us see the different folders and files created in our specified working directory

_R folder

showFolderTree(agrisvy,"_R")

varClassDir(agrisvy) folder

showFolderTree(agrisvy,varClassDir(agrisvy))

preProcScriptDir(agrisvy) folder

showFolderTree(agrisvy,preProcScriptDir(agrisvy))

Template of the R script to be pre-populated

preprocDataDir(agrisvy) folder

showFolderTree(agrisvy,preprocDataDir(agrisvy))

anoScriptDir(agrisvy) folder

showFolderTree(agrisvy,anoScriptDir(agrisvy))

Template of the anonymization to be used to pre-populate all scripts

anoDataDir(agrisvy) folder

showFolderTree(agrisvy,anoDataDir(agrisvy))

anoreportDir(agrisvy) folder

showFolderTree(agrisvy,anoreportDir(agrisvy))

fileDesDir(agrisvy) folder

showFolderTree(agrisvy,fileDesDir(agrisvy))

infoLossReport(agrisvy) folder

showFolderTree(agrisvy,infoLossReport(agrisvy))

tempfileDir(agrisvy) folder

showFolderTree(agrisvy,tempfileDir(agrisvy))

aobDir(agrisvy) folder

showFolderTree(agrisvy,aobDir(agrisvy))

Inlining data

You have the possibility to inline data without opening the script of the corresponding data.

First you need to archive all the datasets (saving them in the agrisvy object)

agrisvyr::ArchiveCleanData(agrisvy)

Let us inline one dataset

library(data.tree)
FromListSimple(agrisvy@cleanData)

library(flextable)
library(magrittr)
head(agrisvy@cleanData$`Season 1_PH`$SEC2_V1_MEMBERS) %>% flextable()

It is possible to do the same for the pre-processed data and the anonymized data with the functions ArchiveAnoData() and ArchiveProcData() respectively