The Institute for Signal and Information Processing

Projects

This page contains an historical archive of ISIP projects. In keeping with our tradition of leadership in public domain software, each project link below points to a web site that contains software, data and educational materials related to the project. If you can't find something, don't hesitate to contact us. To directly view the directory index, click here.

Active Projects | Past Projects

Active Projects

(NSF_FET_QUANT | Started: 2022 | Status: Active | Sponsor: National Science Foundation) FET: MEDIUM: A QUANTUM COMPUTING BASED APPROACH TO UNDIRECTED GENERATIVE MACHINE LEARNING MODELS: We propose a novel way of applying Quantum Annealing Computers (QACs) to training and sampling from Deep Boltzmann Machine (DBM) model distributions. We propose a method that combines QAC and classical sampling that will achieve unprecedented improvements in trainability of undirected probabilistic models on many real-world complex datasets.
(TU_CAT_BIO | Started: 2021 | Status: Active | Sponsor: Temple University) MODELING MOVEMENT THROUGH HETEROLOGOUS BIOFILM STRUCTURES FOR IMPROVED ANTIBIOTIC DELIVERY: We use antibiotics and nanoparticles moving through biofilms as well as mixed species biofilms given the implications for two clinically relevant topics: (1) the treatment of pathogenic biofilms and (2) determining whether heterologous structures in microbiota help prevent the complete eradication of microbiota during antibiotic treatment.
(TU_CAT_DPATH | Started: 2021 | Status: Active | Sponsor: Temple University) AUTOMATIC INTERPRETATION OF DIGITAL PATHOLOGY IMAGES USING DEEP LEARNING: This is a collaborative and interdisciplinary project to detect and characterize cancerous cells in digitized images of pathology slides, while also producing the world’s largest catalog of research-grade digital pathology slides.

Past Projects

(AAUC | Started: 2010 | Status: Completed | Sponsor: Temple University) AUTOMATIC ACOUSTIC UNIT CLASSIFICATION: We investigated automatic derivation of acoustic units for speech recognition using new algorithms such as Particle Swarm Optimization (PSO).
(AE | Started: 2011 | Status: Completed | Sponsor: Temple University) ALTERNATIVE ENERGY DEMONSTRATION: A very successful demonstration of the challenges with generating energy from human powered vehicles. This technology was originally developed for Earth Day at Temple University of April 21, 2011, and consists of a set of bicycles equipped with generators so that electricity can be generated by pedaling the bikes.
(ASL_FS | Started: 2012 | Status: Active | Sponsor: Temple University) AMERICAN SIGN LANGUAGE FINGER SPELLING RECOGNITION: This project investigated the application of nonparametric Bayesian statistical modeling to automatic recognition of finger spelling from single images and video. It is an attempt to extend our work in speech recognition to image recognition and understanding.
(AURORA | Started: 2001 | Status: Completed | Sponsor: ETSI) AURORA EVALUATION OF SPEECH RECOGNITION FRONT ENDS: The goal of this project was to evaluate and compare the robustness of feature extraction algorithms on a large vocabulary task. The target application is cellular telephony. These evaluations are being conducted under the auspices of the Aurora Distributed Speech Recognition working group of The European Telecommunications Standards Institute (ETSI). The Wall Street Journal database (WSJ0) is being used as the basis for experiments.
(AUTO_EEG | Started: 2013 | Status: Completed | Sponsor: University City Science Center and Temple University) AUTOMATIC INTERPRETATION OF EEGS: The goal of this project is to apply machine learning techniques successful in speech recognition to electroencephalography (EEG). A system is being developed to generate a transcription of an EEG signal in real time.
(BANK_HEALTH | Started: 2017 | Status: Completed | Sponsor: Temple University) PREDICTING ENDOGENOUS BANK HEALTH FROM FDIC STATISTICS ON DEPOSITORY INSTITUTIONS USING DEEP LEARNING: Our goal is to assess baseline performance of deep learning systems on operational banking data provided by the U.S. Federal Government (FDIC). Operational data contain many forms of imperfection that make it extremely difficult for the application of traditional deep leaning systems.
(BSE | Started: 2001 | Status: Completed | Sponsor: Mississippi State University) BULLDOG STOCK EXCHANGE: As part of a unique entrepreneurship thrust in MS State's College of Engineering, EE Senior Design teams form companies. These companies are publicly traded on the Bulldog Stock Exchange. This simulation teaches our students about the intimate relationships between technology and business.
(CBN | Started: 2004 | Status: Completed | Sponsor: Mississippi State University) CAMPUS BUS NETWORKING: Networked vehicles are the cornerstone of the next generation intelligent transportation system. In this project, we developed the hardware and software necessary to perform two-way communications with a vehicle track and to collect critical vehicle performance data.
(DIALOG | Started: 2004 | Status: Completed | Sponsor: Mississippi State University) IN-VEHICLE DIALOG SYSTEMS: A voice interface is a superb tool for in-vehicle information access when your hands and eyes are busy. In this project, we are developed a dialog system that provides information about the university and its surrounding area. For example, a user can ask "Where is the nearest restaurant to my hotel?" or "How do I get from the airport to my hotel?".
(DPM_INFERENCE | Started: 2012 | Status: Completed | Sponsor: Temple University) VARIATIONAL INFERENCE ALGORITHMS FOR PHONE CLASSIFICATION: We investigated the use of three nonparametric Bayesian variational inference algorithms for phoneme classification on TIMIT and CALLHOME English and Mandarin.
(HTK_TUTORIALS | Started: 2011 | Status: Completed | Sponsor: Temple University) HTK TUTORIALS: This web site contains detailed tutorials on how to run Cambridge University's Hidden Markov Modeling Toolkit (HTK) on common speech recognition tasks. Complete turnkey systems, along with expected results, are provided, making it very easy for researchers new to HLK to get started.
(HYDRO | Started: 2010 | Status: Completed | Sponsor: Temple University) A MATLAB VISUALIZATION TOOL FOR HYDROLOGY: In this project, we developed an interactive MATLAB-based visualization tool to analyze and model water flow in shallow basins such as streams and rivers.
(IPV6 | Started: 2004 | Status: Completed | Sponsor: Mississippi State University) IP VERSION 6 RESEARCH: IP version 6 (IPv6) is the next generation Internet protocol that has the potential to drastically change the way we use the Internet as part of our everyday lives. We investigated peer to peer IPv6 networks and applications, mobile IPv6, and high performance routing.
(JEIDA | Started: 1996 | Status: Completed | Sponsor: LDC) A JAPANESE COMMAND AND CONTROL WORD DATABASE: The Japan Electronic Industry Development Association's Common Speech Data (JCSD) Corpus is an isolated phrase corpus consisting of 150 speakers (75 males/75 females) and almost 200,000 utterances. It represents an important milestone in Japanese speech recognition technology development. In this project we organized and preprocessed the data so that it was ready for distribution by the Linguistic Data Consortium.
(KS_PREDICTION | Started: 2010 | Status: Completed | Sponsor: Temple University) KEYWORD SEARCH TERM STRENGTH: The goal of this project was to develop a tool that predicts the strength of a keyword search term. We convert a proposed search term to a feature representation and then use these features to predict the reliability of a search term. Key factors include the number of syllables in the word and the phonetic content.
(MOBILE | Started: 2002 | Status: Completed | Sponsor: Mississippi State University) MOBILE COMPUTING USABILITY: Increasingly smaller and more complex computing devices have made the human interface to these systems more critical than ever. The primary goal of this project was to study the design and usability of interfaces to a variety of portable and ubiquitous computing devices.
(NBEST_PRONUNCIATIONS | Started: 1995 | Status: Completed | Sponsor: Texas Instruments) AUTOMATIC PRONUNCIATION GENERATION: Correct recognition of proper nouns is critical to problems in speech understanding and applications involving voice interfaces. We developed a suite of algorithms involving stochastic neural networks, decision trees and other statistical techniques that are capable of automatically generating multiple pronunciations for proper nouns based on only the text-based spelling of the name.
(NEDC | Started: 2012 | Status: Completed | Sponsor: National Institutes of Health) THE NEURAL ENGINEERING DATA CONSORTIUM: This organization develops open source big data resources designed to accelerate progress in machine learning applications in bioengineering.
(NEURONIX | Started: 2015 | Status: Completed | Sponsor: National Institutes of Health) COST-EFFECTIVE CLOUD COMPUTING: This project, which was part of many of our funded research efforts, involved the development of a low-cost extensible heterogeneous compute cluster to support machine learning experiments.
(NIH COHORT | Started: 2015 | Status: Completed | Sponsor: National Institutes for Health) AUTOMATIC DISCOVERY AND PROCESSING OF EEG COHORTS FROM CLINICAL RECORDS: The primary goal of this project is to enable comparative research by automatically uncovering clinical knowledge from a vast BigData archive of clinical EEG signals and EEG reports.
(NPB_ACOUSTIC_UNITS | Started: 2012 | Status: Completed | Sponsor: Temple University) NONPARAMETRIC BAYESIAN ACOUSTIC MODELS: The goal of this project is to apply nonparametric Bayesian approaches to automatically discover subword acoustic units for high performance speech recognition systems.
(NSF CCRI DPATH | Started: 2020 | Status: Completed | Sponsor: National Science Foundation) CCRI: PLANNING: DIGITAL PATHOLOGY RESEARCH CONSORTIUM: This proposal supports a community planning effort for digital pathology focused on planning the data and evaluation resources that will enable high performance, automated interpretation of pathology images using machine learning. We have created a diverse community of digital pathology researchers that contributes data, software tools, and expertise with the communal goal of improving healthcare outcomes through enhanced analysis of biological tissue.
(NSF DPATH | Started: 2017 | Status: Completed | Sponsor: National Science Foundation) MRI: HIGH PERFORMANCE DIGITAL PATHOLOGY USING BIG DATA AND MACHINE LEARNING: The goal of this project is to build a large open-source database of pathology slides that can be used to train high performance deep learning models. The outcome will be a sustainable facility to rapidly collect large amounts of automatically annotated whole slide images.
(NSF_ICORPS | Started: 2015 | Status: Completed | Sponsor: National Science Foundation) AUTOMATIC INTERPRETATION OF EEGS : commercialization of our automated interpretation technology. This project led our team through a process to define the market for EEG technology and focus the technology so that it addresses the greatest customer needs.
(NSF_ITR | Started: 2000 | Status: Completed | Sponsor: National Science Foundation) SPOKEN LANGUAGE INFORMATION RETRIEVAL: Integration of prosodic information, speech recognition and parsing can positively impact the problem of information extraction from spoken documents. This research provided the initial steps towards information extraction from telephone conversations and served as the basis for sophisticated web browsing and data mining.
(NSF_NONLINEAR | Started: 2004 | Status: Completed | Sponsor: National Science Foundation) NONLINEAR STATISTICAL MODELING OF SPEECH: Hidden Markov models (HMMs) have been the primary approach to speech recognition many years. The goal of this project was the development of a new approach to statistical modeling of speech based on nonlinear statistics. We implemented a speaker recognition system using a variety of nonlinear models including linear dynamic models and probabilistic mixtures of autoregressive models.
(NSF PFI-TT | Started: 2018 | Status: Completed | Sponsor: National Science Foundation) REAL-TIME ANALYSIS OF ELECTROENCEPHALOGRAMS IN AN INTENSIVE CARE ENVIRONMENT: In this project, we have developed a real-time version of our state of the art automated seizure detection software. We have also improved performance of the system. The overall goal is to develop technology that is marketable.
(PA-CURE SEIZURE | Started: 2017 | Status: Completed | Sponsor: Pennsylvania Department of Health) ENABLING THE APPLICATION OF DEEP LEARNING TO AUTOMATED SEIZURE DETECTION: Funded by the Commonwealth Universal Research Enhancement Program (CURE), the goal of this proposal is to quadruple the size of the TUH EEG Seizure Detection Corpus by manually annotating three years' worth of EEG data collected at Temple Hospital.
(PMENTOR | Started: 2016 | Status: Completed | Sponsor: Temple College of Engineering) Peer Mentoring: An app that connects undergraduate students with the most appropriate peer mentor. This project was executed as an independent study course by a team of outstanding undergraduates.
(PDOT AT | Started: 2018 | Status: Completed | Sponsor: Pennsylvania Department of Transportation) PennDOT Agent Training: The primary goal of this project is to research, evaluate, improve and develop course material and delivery methods related to training required for authorized agents of PennDOT.
(PDOT HIDT | Started: 2016 | Status: Completed | Sponsor: Pennsylvania Department of Transportation) HIGHWAY INCIDENT TIMELINE DETECTION: The primary goal of this project is to estimate the time it takes for PennDOT to be notified when a highway incident occurs. Emergency response records from counties were manually paired with PennDOT 911 call logs to estimate the time delay.
(POWERTRAINS | Started: 2003 | Status: Completed | Sponsor: Mississippi State University) POWERTRAIN DESIGN AND OPTIMIZATION: State of the art design tools in automotive engineering lacks the power, sophistication, and automation of design tools for the electronics industry. We fundamentally advanced automotive design engineering by introducing optimization and physics-based design principles into standard industry design tools. This allowed designers to globally optimize design criteria such as size, efficiency, cost, weight, volume, and achieve unprecedented reductions in design turnaround time.
(ROBUST_ACOUSTIC | Started: 2004 | Status: Completed | Sponsor: Conversay, Inc.) ROBUST ACOUSTIC MODELING: Field deployment of speech recognition technology results in a number of interesting problems, such as microphone saturation, which severely limit the performance of speech recognition engines. In this project, we studied the effects of microphone saturation and develop algorithms to improve robustness to saturation, clipping, and other forms of signal degradation.
(ROBUST_LOW_PERPLEXITY | Started: 1998 | Status: Completed | Sponsor: MITRE) ROBUST LOW PERPLEXITY VOICE INTERFACES: Robust speech recognition technology for speech recorded and transmitted over narrowband channels requires advances in several components of a speech recognition system: signal processing techniques that produce invariant feature sets; acoustic modeling and training that produce channel-independent acoustic models; noise cancellation techniques that mitigate the effects of impulsive and application- dependent transient noise. This project was a one-year collaboration with the MITRE Corporation that resulted in a prototype of a near real- time system that provided a robust and flexible command and control voice interface in realistic tactical noisy environments.
(SOUTHERN_ACCENTS | Started: 2000 | Status: Completed | Sponsor: Dragon) SOUTHERN-ACCENTED SPEECH: Southern accents are underrepresented in most pubicly available databases. This had led to speculation that performance for such speakers is worse than other better-represented dialects. To test this hypothesis, a small data collection effort was conducted that targeted Southern-accented speakers. Data was collected from February 21 to February 25, 2000. The data collected consisted of a total of 23 speakers (13 males and 10 females) ranging in age from 18 to 56.
(SPEECH | Started: 1998 | Status: Completed | Sponsor: National Science Foundation) INTERNET-ACCESSIBLE SPEECH RECOGNITION TECHNOLOGY: Speech recognition research has always been a core competency in ISIP. Large vocabulary conversational speech recognition LVCSR) is a fascinating technology that draws heavily from the diverse research areas of statistical pattern recognition, digital signal processing, artificial intelligence, linguistics, and information theory. On this web site you will find a powerful and flexible public domain speech recognition system written in C++.
(SUMMER_OF_CODE | Started: 2010 | Status: Completed | Sponsor: Temple University) SUMMER OF CODE: RESEARCH EXPERIENCE FOR UNDERGRADUATES: Every summer we welcome a group of enthusiastic undergraduates into our lab with a goal of developing their software skills and introducing them to fields such as speech and EEG signal processing, machine learning and big data.
(SWITCHBOARD | Started: 1998 | Status: Completed | Sponsor: DoD) SWITCHBOARD RESEGMENTATION: The SWITCHBOARD Corpus (SWB) has become critical to the success of state-of-the-art LVCSR systems. Using this data, however, has not been without its share of drawbacks. Word-level transcription of SWB is difficult, and conventions associated with such transcriptions are highly controversial and often application dependent. By 1998, the quality of the SWB transcriptions for LVCSR was recognized to be less than ideal, and many years of small projects attempting to correct the transcriptions had taken their toll. In February of 1998 ISIP began a project to do a final cleanup of the SWB Corpus, and to organize and integrate all existing resources related to the data into this final release.
(T1_INTERFACE | Started: 1995 | Status: Completed | Sponsor: LDC) A DIGITAL TELEPHONE INTERFACE FOR SUN WORKSTATIONS: Using the Linkon system, a speech data collection board, we developed a fully- expandable, robust system for platform-independent collection of telephone speech data. Our object-oriented software libraries and intuitive GUI provide powerful tools with which even a novice user can efficiently prototype complex applications. Using the system one can generate programs which range from simple single-user prompt/record demonstrations to robust SWITCHBOARD-type multi-user applications.
(TUH_EEG | Started: 2013 | Status: Completed | Sponsor: DARPA, Temple University) EEG CORPUS CREATION: In this project, we are developing the largest EEG corpus ever to be publicly released. The corpus consists of over 20,000 EEGs dating back to 2002. In addition to the raw signal data, metadata about the subjects is available, including medical conditions and treatments. Physician interpretations of the data are also included making this an invaluable resource for machine learning experiments.
(USFS | Started: 1996 | Status: Completed | Sponsor: USFS) SCENIC BEAUTY ESTIMATION OF FORESTRY IMAGES: The United States Department of Agriculture and Forest Services requested we develop an algorithm to automatically determine the scenic beauty of a given forest scene. Their requirement is a consequence of rising public concern to preserve forest beauty. To achieve this, we have developed an extensive database that can support our algorithm development. The database consists of 637 unique images, each image having various subjective ratings for their scenic beauty content. The database extensively samples several dimensions of the problem including year, season, time of day, angle and treatment. In order to automatically relate the beauty of an image to the subjective beauty ratings, we developed algorithms to extract features from the image that determine its scenic beauty.
(VOICE_ANALYSIS | Started: 2003 | Status: Completed | Sponsor: Creare, Inc.) COGNITIVE ASSESSMENT USING VOICE ANALYSIS: The goal of this project was to design an effective fatigue monitoring and assessment system by characterizing changes in a human voice as a speaker becomes fatigued or stressed. A remote, near-real-time assessment system to monitor the fatigue levels of military personnel was developed.
(VPMS | Started: 2004 | Status: Completed | Sponsor: Mississippi State University) VEHICLE PERFORMANCE MONITORING SYSTEM: This project is a one-year collaboration with the Mississippi Department of Transportation (MDOT) to adapt and apply the Mississippi State University wireless web-based vehicle performance and monitoring system (VPMS).
(WE2 | Started: 2014 | Status: Completed | Sponsor: National Science Foundation) BRAINWAVE RECOGNITION: This project is a half-day laboratory on brainwave recognition that includes an introduction to Python. It was originally developed as part of our outreach program for Temple University's Women in Engineering Summer Program.