I’m wondering what kind of bilingual speech corpora there are for training purposes. The Fisher-Callhome corpus is widely used, but as somebody who just wants to play and build a speech translation system, its $500 fee is too high.
Has anybody used any other bilingual speech corpora that is reasonably sized?
submitted by /u/HXSC
[link] [comments]
The housing market is complex. There is a continuously changing supply of student housing units around any given education campus. Moreover, the accepted value of a unit continuously changes based on physical and social variables. These variables could include proximity to campus with regard to other available options, friend groups living nearby, and the availability of nearby parking as other properties fill. The interplay happens at all levels—entire properties may shift in value and specific units within them may exacerbate or counteract those shifts.
For a property management company to earn the maximum revenue from its rental units, it needs to price each unit just within the price-point for the tenants—but it doesn’t know what their price constraints are. The company would not want to leave money on the table by setting a price too low. Setting a price too high can mean that the unit sits empty—effectively costing the company to maintain the unit. Finding that balance is a difficult problem.
Entrata, a comprehensive technology provider of multifamily property management solutions, solves this problem by employing machine learning (ML) with AWS. Specifically, they feed location-specific and even building-specific data (such as occupancy, proximity to campus, and lease term length) into an ML-based dynamic pricing engine running on Amazon SageMaker. The model helps Entrata’s customers—property managers—to predict occupancy levels and in turn optimize their prices of student housing.
At the implementation level, this solution relies on a number of AWS offerings. AWS Glue extracts Entrata’s historical data into Amazon S3. This data enables Amazon SageMaker to make pricing predictions, which are written to an output bucket back into Amazon S3. Entrata’s applications consume this data request using API Gateway, which triggers AWS Lambda functions to deliver the most relevant forecast for any available unit.
Entrata developed this solution in partnership with AWS Premier Consulting Partner 1Strategy, a Seattle-based consultancy that helps businesses architect, migrate, and optimize their workloads on AWS. The partnership between 1Strategy and Entrata has existed for years, but the ML work is their most recent—and arguably, most impressive—joint technical accomplishment.
Their collaboration previously focused exclusively on data management through AWS—which in itself proves a non-trivial challenge due to the location, size, and complexity of the data. Entrata currently serves greater than 20,000 apartment communities nationwide and offers a variety of tools, from mobile apps to lease signing portals to accounting platforms.
The novel ML solution is exciting. Entrata’s CTO, Ryan Byrd, says, “The impact is far ranging and positive. Automating back-office functions with Amazon ML frees property management to focus on people first, instead of performing rote behind-the-scenes guessing of price recommendations.”
Entrata plans even more work with AWS in the future. Byrd adds, “AWS technologies will decrease our time to market with various ML projects.” He and his colleagues on the Entrata team are keen to aid customers in their decision-making efforts. They also use ML for various operational elements for their and their customers’ businesses, strategic planning, and maintenance management.
Marisa Messina is on the AWS ML marketing team, where her job includes identifying the most innovative AWS-using customers and showcasing their inspiring stories. Prior to AWS, she worked on consumer-facing hardware and then university-facing cloud offerings at Microsoft. Outside of work, she enjoys exploring the Pacific Northwest hiking trails, cooking without recipes, and dancing in the rain.
I’m trying to build a pipeline to detect the presence of a compound in spectroscopic data. I have ~100 samples that consist of vectors of size 1000 representing a specific frequency band. At first, I was thinking of using a CNN because I’m trying to detect patterns, but these patterns are location dependent since they represent absorption at a given frequency. I’m not sure where to go from here so any advice is appreciated.
submitted by /u/Clssq
[link] [comments]
Link to competition: Traffic4cast — Traffic Map Movie Forecasting
Predict high resolution traffic flow volume, heading, and speed on a whole city map looking 15 minutes into the future! Kicking off a series of annual competitions, this year’s data is based on 100 billion probe points from 3 cities mapped in 5 minute intervals, showing trends across weekdays and seasonal effects. Improved traffic predictions are of great social, environmental, and economic value, while also advancing our general ability to capture the simple implicit rules underlying a complex system and model its states.
Join us for access to challenge Contest data data and code.
We provide a unique data set derived from trajectories of raw GPS position fixes (consisting of a latitude, a longitude, a time stamp, as well as the vehicle speed and driving direction recorded at the time). The data is made available by HERE Technologies and originates from a large fleet of probe vehicles which recorded their movements in multiple culturally and socially diverse metropolitan areas around the world throughout the course of an entire year. The overall number of raw probe-points that we share with the scientific community is based on the unprecedented number of over 1011 probe-points, corresponding to over 300,000 frames, which at a 24 frame/s play rate would give in excess of three hours data-movie footage.
To assist you getting up and running, IARAI provides a number of useful utilities, Loader code, and some Baseline Models through the Traffic4cast project at Github. The project is open to contributions, so please feel free to share your code, too! Once you have first results, you can test them in the Core competition by submitting your predictions, with scores available within an hour on the Core leaderboard. The Submission system will be open for submissions from 15 July until 15 October. The three top-ranked teams in the core competition leaderboard are honoured at NeurIPS and receive up to $10,000 cash value prizes. Special awards include fully funded 12-month Research Fellowships at IARAI in Vienna. Please use the Competition Forum for discussions!
We look forward to your contributions and seeing you in Toronto.
For the Organizing Team at IARAI,Sepp Hochreiter, David Kreil, and Michael Kopp
Key Dates: 15 Jul … Leaderboard live
15 Oct … Submission deadline
30 Oct … Award announcement, symposion invitations
7 Dec … Traffic4cast symposion Toronto
8 Dec … NeurIPS
submitted by /u/iarai_competition
[link] [comments]
Group lasso in Python
I recently wanted group lasso regularised linear regression, and it was not available in scikit-learn. Therefore, I decided to create my own little implementation of it and I ended up becoming borderline obsessive on figuring out how to do it properly.
Here is the github link: https://github.com/yngvem/group-lasso
This my first publicly shared open-source project so I’d be delighted for any feedback you might have.
Information about the package
Group lasso is a regularisation algorithm used in statistics/machine learning/data science when you have several measurements from different sources and want only a few of the sources to be used in prediction. Also, this implementation is FAST. I am currently sitting on a moderately priced laptop and I can fit models with 10 000 000 rows and 500 columns without it struggling.
User guide
The package can easily be pip-installed by typing pip install group-lasso. After that it’s as simple as creating a GroupLasso instance and calling the GroupLasso.fit(X, y) method. A full description is in the readme at GitHub.
Future work
I am currently working on implementing the same update scheme for logistic regression. I have currently done this for the one-class sigmoid based logistic regression, and it seems to work. The next step is implementing it for the multi-class softmax based logistic regression and testing it on some datasets. This all takes some time, since I need to derive some mathematical constants for the optimisation algorithm to work (Lipschitz bounds of the gradient to be specific).
There are other parts that I am working on as well. I should probably have support for Python 3.5, not just 3.6 and I think that should be fixable if I only remove the f-strings and the underscores in the numbers. I also hope to get Sphinx documentation up and running, but that will probably be for after the summer.
Another facet for future work is support for sparse matrices. I don’t think that should be too difficult, but I haven’t worked much with them till now so I don’t know how much of a problem that will be.
Mathematical background
Solving the group lasso problem involves solving an optimisation problem that in some senses are difficult (for the interested: it is non smooth, but luckily convex). Normally the group lasso problem is solved using an algorithm called block-coordinate descent, which can be slow. Therefore, I implemented the update scheme for a newer optimisation algorithm called FISTA.
submitted by /u/yngvizzle
[link] [comments]
I’m a PhD with publications at top conferences. I have 2 years of experience leading NLP/RL projects(team size generally 2-3). A company that just finished their series A is asking me to manage their machine learning team(5 people). Roughly what should I expect as fair compensation(based out of San Francisco)?
submitted by /u/ElectronicSet0
[link] [comments]
When a Table is moved from server A to B, errors are expected to creep in. What is the fastest way to find how close are 2 Tables (or matrices)?
submitted by /u/gursi1
[link] [comments]
I have ~500GB of extremely sparse, tabular data with millions of features and a single binary target. I have been training FFM models with https://github.com/cttsai1985/libffm and they take around 6 hours to train. This model creates millions of cross-features on top of the millions of original features and trains parameters for all of them.
As complex of a model as is it, that FFM model only takes 6 hours to train and converge after ~10 epochs on 500GB data on a single machine with ~ 8 cores. On the other hand, training an extremely simple tensorflow model on this dataset (embedding layer -> sigmoid output neuron) takes almost 2 days for a single epoch on this dataset. I’ve optimized everywhere I can think of, using tf.data api, etc, it seems like tensorflow is just really slow compared to FFM. What’s the deal?
submitted by /u/DstnB3
[link] [comments]
Is this what many/most/some are using to do distributed training today ? What are alternatives and why are they better ?
https://arxiv.org/abs/1106.5730
Thanks!
submitted by /u/TalkingJellyFish
[link] [comments]