Darrell Taylor

Transitioned from Contractor to Permanent with a view to introducing better engineering principals in the Data Science team. The main brief for this role was to to put tools and processes in place to enable the Data Scientists to deliver models to the business faster. * Custom Jupyter Notebook Docker image to aid local off-line development and proof of concept of PySpark applications. * pytest backed PySpark framework to allow Data Scientists to build applications in a test driven manner. * Built Jenkins CI pipelines integrated with Stash to facilitate automated test and release processes. * RStudio Docker image built with connectivity to the main production cluster, allowing Data Scientists to develop models against live data with SparkR. Deployed on the production Rancher cluster. * Neo4j Docker image with Impala integration allowing users to analyse data using the graph database. * Neo4j Docker image to provide a custom Data Dictionary, providing a graph database of table, pipeline, model, and Jenkins relationships. * Deploying models and research produced by Data Scientists. Essentially applying good engineering practice to code from Data Scientists to allow it to run at scale in a cluster. Update 05/2018: It seems this role is what some people would call a Machine Learning Engineer

Contracted to help across the board with a green field project to migrate away from an existing Oracle Data Warehouse which was unable to scale up to the demands of the business without incurring considerable expense. The initial phase of the project was highly exploratory, building reference pipelines to ingest data from several Informix source. After three years the ETL processes are orchestrated by an in-house Ruby based DSL language that meets all the needs of the problems encountered when building the early reference pipelines. The business successfully turned off the old Oracle Data Warehouse in March 2016. * Liabilities simulation. Implemented a Monte Carlo simulation in Hive to predict liabilities for all unsettled bets in the business. * Data design for many of the foundation tables in the cluster. * PySpark implementation for cleaning up telephone numbers. http://engineering.skybettingandgaming.com/2016/03/30/pyspark-phone-numbers/ * Data design and optimisation of Impala based queries, taking full advantage of columnar storage in Parquet files. * Proof of Concept Data Dictionary and Data Lineage application using the Neo4j graph database. * Advising across the full stack, from hardware and configuration all the way through to Data Visualisation. * Strong hands-on experience with all of the follow technologies : Hive, Hbase, MapReduce, YARN, Sqoop, Impala, Parquet, Spark, PySpark, Cloudera, Informix, Oracle, Jenkins, Ruby, Neo4j.

Largely responsible for building and maintaining the Localphone platform from the ground up. In the early phase of Localphone this involved writing large parts of the website framework, integration with the VoIP platform, server and database administration, customer and technical support, data centre build outs, and anything else that needed doing within the business while in startup mode. As the business grew extra responsibilities included mentoring and training of new developers, and sysadmins, more in-depth work around the VoIP platform and call accounting systems. Recently responsible for research development and build out of the companies Hadoop platform for call accounting and reporting purposes.

Developing telecoms billing verification software.

Rewrite and enhance existing in-house systems which managed all aspects of the design and project management processes within the business