In Project Talia at Microsoft Research Cambridge, we are exploring how we can best leverage AI to help improve the effectiveness of important mental health services. Specifically, in partnership with SilverCloud Health, we develop machine learning tools to enhance digital mental health services that deliver cognitive-behavioral treatment (CBT) programs to a large and growing number of people in need of effective care.
A key impediment to reinforcement learning (RL) in real applications with limited, batch data is in defining a reward function that reflects what we implicitly know about reasonable behaviour for a task and allows for robust off-policy evaluation. In this work, we develop a method to identify an admissible set of reward functions for policies that (a) do not deviate too far in performance from prior behaviour, and (b) can be evaluated with high confidence, given only a collection of past trajectories. Together, these ensure that we avoid proposing unreasonable policies in high-risk settings.
Applying machine learning tools to forecasting adverse events in intensive care can be invaluable in providing clinicians with the time needed to intervene and improve patient outcomes. In this work, we describe an end-to-end approach to the prediction of hypotension from critical care data using off-the-shelf classification models. Evaluated with standard performance metrics, these models appear to learn effectively from available data, with additional multi-modal information improving classification accuracy. However, we show that this improvement is thrown into question when probing further into medical context and choices in data curation, highlighting the need for domain-centric evaluation.
Laboratory testing is an integral tool in the management of patient care in hospitals, particularly in intensive care units (ICUs). There exists an inherent trade-off in the selection and timing of lab tests between considerations of the expected utility in clinical decision-making of a given test at a specific time, and the associated cost or risk it poses to the patient. To this end, we develop and extend principles of Pareto optimality to improve the selection of actions based on multiple reward function components while respecting typical procedural considerations and prioritization of clinical goals in the ICU. Our experiments show that we can estimate a policy that reduces the frequency of lab tests and optimizes timing to minimize information redundancy.
The management of invasive mechanical ventilation, and the regulation of sedation and analgesia during ventilation, constitutes a major part of the care of patients admitted to intensive care units, but clinical opinion on the best protocol for weaning patients off of a ventilator varies. This work aims to develop a decision support tool that uses available patient information to predict time-to-extubation readiness and to recommend a personalized regime of sedation dosage and ventilator support. To this end, we use off-policy reinforcement learning algorithms to determine the best action at a given patient state from sub-optimal historical ICU data.
Pixie, a socialbot submitted to the 2017 Alexa Prize, was developed with the objective of holding engaging casual conversations on open-ended topics. We develop a modular, heterogeneous architecture, and highlight the successes and shortcomings of our primarily template-based approach. We reflect on our findings on alternative approaches attempted in the prototyping phase, in particular the challenges of bridging breakthroughs in deep learning with the frontier of machine social dialogue.
We consider a dataset of 2-photon excitation microscopy image sequences recorded through a cranial window in GCaMP transgenic mice, showing V1 neuronal activation patterns over time (through increased GFP fluorescence). Analysis of this data requires distinguishing neurons in the image focal plane from other objects, e.g. blood vessels, dendritic branches, and neurons fluorescing above and below the focal plane. The task of neuron identification and boundary detection is currently performed manually but is potentially amenable to automation through various machine learning approaches, such as SVMs or convolutional neural nets. We find that both approaches yield comparable results, and perform well enough to considerably speed up neuron labelling.
This work investigates the scaling properties of Recurrent Neural Network Language Models (RNNLMs) We discuss how to train very large RNNs on GPUs and address the questions of how RNNLMs scale with respect to model size, training-set size, computational costs and memory. We train the largest known RNNs and present relative word error rates gains of 18% on an ASR task. We also present the new lowest perplexities on the recently released billion word language modelling benchmark, 1 BLEU point gain on machine translation and a 17% relative hit rate gain in word prediction.
This project involves the design and development of an adaptive speaker recognition algorithm, tailored for implementation in the Nao domestic service robot for the Chinese market. The problem is fundamentally text independent open set identification; the proposed algorithm distinguishes between a small set of known speakers, and identifies a new speaker as unknown. It is designed to operate in real time, be robust to both background disturbances and internal robot noise, and adapt to changes in the voices of the registered speakers.
