HOUSINGNEWS REPORT
SPLITTING THE ATTOM
that contribute to the universe of data that directly or indirectly impacts the U.S. housing market. IOT sources are evolving rapidly and include home automation, utility telemetry, communication device wearables and other web-enabled devices yet to even be considered. The expanding universe of data sources challenges traditional approaches to how data solutions are developed and supported within an organization. Typical solutions included identification
of the required data sets, sourcing the data sets from various flat files, building separate ETL (extract, transform and load) processes for each data set to integrate the source data into a common database from which a solution could then be constructed. Concurrently with the ETL build, the IT team would have to devise a plan to house the data and expand server resources to accommodate the new workload. This practice works when there are three or four sources to
manage and the total size of the data does not require exorbitant hardware acquisition to accommodate the compute or storage requirements. However, what if a research project requires the compilation of years of air quality and property valuation data to evaluate the correlation of air quality levels and home values over time? Granular air quality measurement data and monthly valuation data for each parcel in the United States constitutes
EXISTING DATA FI LE PROCESS
3. Review vendor’s data catalog to determine table structire for housing the data
1. Connect to partner’s ftp
2. Download data files
4. Create table structures
6. Resolve any incongruities in data/record layout
8. Map data to internal applications
5. Load data to staging server
7. Evaluate field populations, row count, and depth of data
9. Build software to load data
10. Test/QA process with live data
11. Fix bugs in software
12. Deploy code
13. Data ready to use
ATTOM DaaS PROCESS
1. Connect to partner’s ftp
2. Data ready to use
23
JULY 2017 | ATTOM DATA SOLUTIONS
Made with FlippingBook Online newsletter