The hardware and bandwidth for this mirror is donated by dogado GmbH, the Webhosting and Full Service-Cloud Provider. Check out our Wordpress Tutorial.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]dogado.de.
get_statcan_geographic_attributes() now reads parquet
files hosted under
https://data.mountainmath.ca/census/geo_attributes/ instead
of downloading from Statistics Canada, which blocks programmatic
downloads of these files (#228, #233). This requires the
duckdb and DBI packages. Column names
are now simpler and follow one convention across census years, so code
using the previous names needs updating, e.g. CTCODE_SRCODE
is now CTCODE for 2021, and DBuid and
DAlat are now DBUID and DARPLAT
for 2006 and 2011. To make up for the simpler names, columns now have
labels describing them in the language of choice, set by the new
language parameter. Counts are now integers and coordinates
are numeric. Data gets downloaded again once. The previous cache in the
attribute_files folder of the cache directory and any
copies of these files in the statcan_downloads folder are
no longer used, and the function shows a command to delete them once per
sessionget_statcan_geo_suite() now reads parquet files hosted
under https://data.mountainmath.ca/census/geo_suite/
instead of downloading from Statistics Canada, which blocks programmatic
downloads of the GeoSuite (#228, #234). This requires the
duckdb and DBI packages. Column names
follow the attribute files from
get_statcan_geographic_attributes(), so code using the
previous names needs updating, e.g. DAuid_ADidu is now
DAUID. The English part of the previous name of each column
is in the source_field entry of the columns
metadata,
e.g. attr(x, "metadata")$columns$DAPOP2021$source_field is
"DApop_2021". Counts are now integers and representative
points numeric, previously these were returned as text. The previous
cache in the geo_suite_<year> folders of the cache
directory and any copies of the GeoSuite in the
statcan_downloads folder are no longer used, and the
function shows a command to delete them once per sessionget_statcan_geography_relationships() is deprecated and
will be removed in the next release, as Statistics Canada blocks
programmatic downloads of the relationship file (#228). It warns once
per session. get_statcan_geographic_attributes("2021") has
the same DGUIDs for each dissemination block, except for census
agricultural regions, which are only in the relationship file. Unlike
the relationship file, the attribute file fills CMADGUID
outside census metropolitan areas and census agglomerations with the
metropolitan influenced zone, and CTDGUID outside areas
with census tracts with a placeholder census tract. Use
CMATYPE %in% c("B", "D", "K") instead of
!is.na(CMADGUID) to select the blocks in census
metropolitan areas and census agglomerationsget_statcan_geo_suite() now covers the 1996 to 2021
censuses and all tables of the GeoSuite, including census consolidated
subdivisions (CCS), the parts of areas spanning provinces
or other areas (_P tables) and the code tables like
CSDTYPE (#234). Called without level it lists
the tables of a census year. Like
get_statcan_geographic_attributes(), it gains the
language and quiet parameters, columns have
labels in the "label" attribute, and the
"metadata" attribute of the result has the attribution,
licence, source and notes on each tableget_statcan_geographic_attributes() now covers the
1981, 1986, 1991, 1996 and 2001 censuses (#233). 1981 to 1996 are at the
enumeration area level, 2001 was derived from the 2001 GeoSuite. Columns
have labels in the "label" attribute, in the language given
by the new language parameter, and the
"metadata" attribute of the result has the attribution,
licence, source and notes on each file. With refresh = TRUE
it reports whether the hosted file has changedget_census(),
list_census_vectors(),
parent_census_vectors(),
child_census_vectors(), find_census_vectors(),
visualize_vector_hierarchy() and
dataset_attribution() gain a language
parameter, one of "en" or "fr". It defaults to
the new cancensus.language option and otherwise to
"en", so options(cancensus.language = "fr")
switches the language for a session. Column layouts are the same in both
languages, and in get_census() changing the language does
not download the data again or use up API quota. Vectors without French
labels, currently the tax and cross-tabulation datasets, fall back to
English with a warning once per session and dataset. Region names and
the type, units and aggregation
columns of list_census_vectors() stay as they arelanguage parameter where a function
has one, otherwise the cancensus.language option, and
otherwise the language of the system locale,
e.g. LANGUAGE=fr. Data labels still default to English
unless the option is set, so that scripts give the same results on all
machines. Conditions have classes of the form
cancensus_<id>,
e.g. cancensus_no_api_key, along with
cancensus_error, cancensus_warning or
cancensus_message, so they can be caught independent of the
language. Errors from the CensusMapper API and other packages are passed
on as they are.get_census() gains labels = "labelled"
(#230), which names columns by vector like labels = "short"
and stores the label of each variable in the "label"
attribute of its column. The other columns get short names, like
Name, Area, Quality and
Population_Previous, that are the same across datasets and
for data and geometry requests, along with the same types and column
order. Columns that data and geometry requests both return, like the
region name and area, only appear once. These columns also get labels in
the requested language, including the census year where that applies,
e.g. Population is labelled Population 2021
for CA21. Tax and cross-tabulation datasets use the year of the census
they take these columns from, e.g. Population 2016 for
TX2018. Columns named by naming vectors get the label of
their variable, and if that name is already used by another column, that
column gets a trailing underscore, e.g. Population_ for
vectors = c(Population = "v_CA21_1").
labels = "detailed" and labels = "short" do
not changelabels parameter of
get_census() is planned to change to
"labelled" in a future version. It now defaults to the new
cancensus.labels option and otherwise still to
"detailed", and get_census() shows a message
once per session if neither labels nor the option are set.
Setting either one, e.g. with
options(cancensus.labels = "detailed"), ensures code keeps
working when the default changes, and quiet = TRUE
suppresses the messageget_census() now stops with an error for invalid
labels values, previously these were treated like
labels = "detailed"cancensus/<version> httr/<version> R/<version>.
This lets the API know which tools are making requests and retain
backward compatibility with older package versions as the API
evolvesoptions(cancensus.user_agent_prefix = "mypackage/1.2.3"),
which is prepended to the user agentget_census(),
get_intersecting_geometries() and the StatCan download
functions no longer emit this notice otherwise, and
quiet = TRUE suppresses it where available. Since this is a
warning, it will show up in rendered documents and turns into an error
under options(warn = 2), setting a cache path avoids
itset_cancensus_cache_path()
without install = TRUE to set up a project-specific
cacheRetry-After header, even with the default
retry = 0 (#235). This applies to
get_census(), get_intersecting_geometries(),
list_census_datasets(), list_census_regions(),
list_census_vectors() and the check for recalled data. If
the server stays busy, the error now says so instead of reporting a
generic download failure. With retry greater than zero,
other transient failures are retried as before, except that HTTP 500
errors are no longer retried, as they now point to a problem with the
request rather than server loadlist_census_datasets() and
list_census_regions() now respect the
cancensus.base_url option like all other CensusMapper API
callslist_census_datasets() no longer reads its list from disk
on every callget_census() now accepts "CA" and
"CMACA" in addition to "CMA", both for the
level parameter and as names of regions. All
three refer to the same level, which contains both census metropolitan
areas and census agglomerations. This also fixes the error when passing
census agglomerations from list_census_regions() through
as_census_region_list() to get_census()
(#208)show_cancensus_api_key() and
show_cancensus_cache_path() no longer run when building
documentation, which printed the values set on the build machine into
the generated pageslabel_vectors() now works for sf and
sp results of get_census(). Renaming columns
dropped the labels for most requests with geographic datafind_census_vectors(query_type = "keyword") no longer
errors in non-interactive sessions when there are additional matches
with less precision. It returns the top matches with a message about the
number of additional matches, since the prompt to show them can’t be
answeredlabel_vectors() now returns the full label for vectors
with colons in their label,
e.g. Size of census family: 2 persons in CA06 and CA11.
Previously the Detail column was cut off at the second
colon, and the result had a third unnamed columnfind_census_vectors() keyword and semantic search no
longer drop non-ASCII letters from the queryget_census() no longer errors when only some of the
vectors are named, the unnamed ones keep their vector code
as column nameget_census() fail for users
with a non-empty cacheget_census() and list_census_regions() no
longer trigger a deprecation warning with readr 2.2.0 or newer when
parsing downloaded data, cancensus now requires readr
2.0.0 or newer.data, which is
deprecated in tidyselect 1.2.0 and newershow_cancensus_cache_path() now also reports a cache
path set via the cancensus.cache_path option, previously it
reported that no cache path was setFALSEget_statcan_geographic_attributes() no longer returns
misaligned columns for 2006 and 2011, the fixed-width layout it used did
not match these files. For 2006 the columns from DAlamy on
were wrong, for 2011 the columns from CDuid on, so results
based on these columns from earlier versions should be checkedget_statcan_geographies() and
get_statcan_geography_relationships() now fail gracefully
when Statistics Canada blocks the download with a Cloudflare browser
verification (#228). They return NULL with a warning that
gives the link for downloading the file in a browser and the location in
the cache directory to move it to, and pick up the file from there on
the next run. If the download was blocked while refreshing existing
cached data with refresh = TRUE, they return the cached
data with the same warning instead of NULL. The file has to
stay zipped, so on macOS the warning points out that Safari unzips
downloads automatically, and it flags folders that look like an unzipped
copy of the fileget_intersecting_geometries() with
simplified = TRUE now returns the character vector of
region ids, previously it returned them collapsed into a single string.
The documentation of simplified had the two cases the wrong
way around and has been correctedget_census() now checks the resolution
parameter when geometries are requested, previously it was only checked
when it was not usedget_census() now stops with an error pointing to
get_statcan_wds_data() for the "ADA" level,
which CensusMapper does not have data for. Previously this warned about
an unknown level and returned regions without dataget_statcan_geographies() no longer leaves empty
temporary folders in the cachefind_census_vectors() with
query_type = "keyword" no longer returns all vectors for
queries without any letters, e.g. "2021"set_cancensus_api_key() and
set_cancensus_cache_path() with install = TRUE
now keep the previous version of the .Renviron file as
backup, previously the backup was only made the first time and never
updated. They only replace the line setting the key or cache path, and
no longer remove other lines that mention CM_API_KEY or
CM_CACHE_PATH, and they no longer append to the last line
of a .Renviron file that does not end in a line breakget_statcan_wds_metadata() only caches metadata that is
valid, previously an error page or an interrupted download got cached
and failed on every call until refreshed.
get_statcan_wds_data() gives an informative error for
DGUIDs it can’t determine the geographic level of, and explains failed
certificate checks of the Statistics Canada server as intendedget_statcan_geographies() and
get_statcan_geography_relationships() for blocked downloads
and invalid manually downloaded files now have the classes
cancensus_statcan_blocked and
cancensus_statcan_invalid_manual_file, like all other
conditions of the packageget_intersecting_geometries() gives an informative
error for geometries without a coordinate reference systemset_cancensus_api_key() and
set_cancensus_cache_path() with
overwrite = TRUE, install = TRUE no longer truncate the
user’s .Renviron when it does not already contain the
corresponding keyget_statcan_wds_data() no longer caches error response
bodies as data, and refresh = TRUE now bypasses the data
cache as documentedremove_recalled_cached_data() no longer over-matches
recalled vector IDs (e.g. a recall of v_CA21_1 no longer
also flags cached data for v_CA21_10)get_census() now warns about recalled data when reading
tabular data from the local cache, matching the geometry pathparent_census_vectors() and
child_census_vectors() now stop with “Unable to determine
dataset” for mixed-dataset input instead of silently dropping
vectorsfind_census_vectors() exact search now matches queries
containing regex metacharacters (e.g. "income ($)")
literally instead of erroring or silently failingfind_census_vectors() semantic search now considers the
best match for every query word (previously only the full phrase), and
restores suffix n-grams dropped in 0.6.0find_census_vectors() keyword search no longer matches
every vector when the query starts with a digit or punctuationvisualize_vector_hierarchy() accepts multi-vector
character input as documented, and no longer labels nodes truncated by
max_depth as leaves (now marked with ...)add_unique_names_to_region_list() now correctly
preserves grouping of grouped inputlist_census_datasets() and
list_census_regions() staleness warnings now respect
quiet = TRUERetry-After headerslist_census_vectors()
and list_census_regions(): repeated calls within a session
no longer re-read and deserialize the cached file on every access (cache
hits are ~1000x faster)parent_census_vectors() and
child_census_vectors() use a hash-based BFS traversal, 4-8x
faster on deep hierarchies on top of the 0.6.0 improvements, with
identical outputfind_census_vectors() semantic search prunes
fuzzy-match candidates by length bound before computing Levenshtein
distancesget_census() no longer scans the package library
(installed.packages()) on spatial callsget_statcan_wds_metadata() parent ID extraction is
vectorized (~5-10x faster metadata parsing for large levels such as
DA)parent_census_vectors() is now 1.9x
faster (92% speedup) by caching the full vector list once
instead of repeated lookupschild_census_vectors() is now 1.2x
faster (23% speedup) with same optimizationfind_census_vectors()
tests/testthat/ directory structuremicrobenchmark and testthat to
Suggestsvisualize_vector_hierarchy())list_census_regions()sf package
when user requests spatial data but does not have the required package
installed as opposed to erroring out.sf package
when user requests spatial data but does not have the required package
installed as opposed to stopping with an error.get_census call that produces
a warning if locally cached data has been recalled by Statistics
Canadaget_censussf compatibility and
agr attributekeep_parent when calling
child_census_vectors() that retains the input parent
variable in the list of result. We found that in many cases user would
follow up a call to child_census_vectors() with a
bind_rows(...) to do this, so this should save a step.t column not present in downloaded
data.get_intersecting_geometry function to take
advantage of new Censusmapper API endpoint. Check out the new vignette
detailing how to use it.set_api_key,
set_cache_path, show_api_key, and
show_cache_path added.find_census_vectors()find_census_vectors() and deprecation of
`search_census_vectors(). See the Data
discovery: resources for finding available and relevant data
vignette for additional information.list_census_regions()list_census_datasets()explore_census_regions and
explore_census_vectors which open a browser page towards
the interactive discovery and selection tools on the Censusmapper websiteattribution_for_dataset which provides
accurate attribution information for citation and visualizations for a
given dataset.get_census_geometry() is now hard-deprecated and will
stop the program flow. Use get_census() instead.sp and
sf spatial objectsgeojsonsf dependencymax_leaves_option for the
search_census_vectors function. Adds functionality to set
maximum depth for child census vectors, i.e. max_level=NA
as an additional parameter. Then e.g. max_level=1 would
only get direct children and no grandchildren.get_census_geography is now soft deprecated and rolled
into the standard get_census with parametersreadr packagesf and
ggplot2 packagessftempdir()options(cancensus.cache_path = 'XXX')sf-class data frames if
option sf was selected for spatial data.read.csv was incorrectly loading text
data if readr::read_csv was not availableNEWS.md file to track changes to the
package.list_census_vectors() changed to
have quiet = TRUEThese binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.
Health stats visible at Monitor.