Large Scale Analytics on Scientific Image Data

Download or read book Large Scale Analytics on Scientific Image Data written by Parmita Mehta and published by . This book was released on 2020 with total page 123 pages. Available in PDF, EPUB and Kindle. Book excerpt: Scientific discoveries are increasingly driven by analyzing large volumes of data. Advances in data collection and storage technologies, availability of cloud compute resources, and better algorithms and readily available open-source libraries are responsible in equal measure for this phenomenon. Large proportion of scientific data is in form of images as many scientific instruments such as telescopes, microscopes, satellites, x-rays, MRIs, etc. produce data in image formats.However, commercial systems have paid scant attention to scientific image analysis workloads and as a result scientists working with images spend a lot of effort building bespoke and often fragile support for such analyses.In this dissertation, we first evaluate several popular systems on scientific image analysis work-loads. We then perform an in-depth image analysis, which yields novel results in ophthalmology.Finally, we use our findings to propose a novel technique to ease some of the data management burden associated with scientific image analysis, specifically debugging of deep neural networks.Specifically, first we assess existing big data systems and frameworks for suitability of scientific image analyses workloads. We evaluate five representative systems (SciDB, Myria, Spark, Dask, and TensorFlow) both qualitatively (ease of use) and quantitatively (scalability and performance)on two real-life image analysis use cases from astronomy and neuroscience. We find that each of them has shortcomings that complicate implementation or hurt performance.Next, we propose a new, comprehensive, and more accurate ML-based approach for population- level glaucoma screening. In this project we embed ourselves in the process of scientific discovery by analyzing a publicly available large dataset to further the state of art in ophthalmology. Our model is highly accurate (AUC 0.97) and interpretable. It validates biological features known to be related to the disease, such as age, intraocular pressure and optic disc morphology. Our model also points to previously unknown or disputed features, such as pulmonary capacity and retinal outerl ayers. Finally, we utilize lessons from building interpretable deep learning models for automated glaucoma detection to propose a novel sampling technique for deep learning model diagnosis. Our experience demonstrated that scientists utilizing deep learning often spend majority of their time managing the data associated rather than focusing on science. Our sampling technique seeks to reduce the data management burden for scientist working on such analyses, making the process of deep learning model diagnosis simpler and more efficient.