Can we derive formulas from neural networks or decision trees?

(updated on June 10, 2020)

I was wondering if we could derive numerical relationships between input variables and output variables from nonlinear model structures such neural networks and decision trees. I find a Q&A on ResearchGate, and I like the answer.

One answer wrote: "It is possible to obtain an equation after developing an ANN for prediction. in fact, you will end up with a long equation including the inputs, weights, biases, etc."

Multi-output Regression Models

(updated on June 10, 2020)

A multi-output regression task predicts multiple numerical properties for each sample (reference).


The article titled "Regression Models with multiple target variables" by Kiran Karkera (link) covers exactly what I am interested. Here are the key points other than the modeling details.

  1. Terminology: multi-output regression or multi-target regression; related terms for classification tasks are multi-label classification, multi-class classification, and multioutput-multiclass classification (aka multi-task classification).
  2. Popular open source ML libraries have little support for the multi-output regression task.

These are the two papers that are mentioned in Kiran Karkera's article.


Here is an article discussing how to develop multi-output regression models with python posted online on March 27, 2020.

 

2020 OR Talks

(update on April 20, 2020)

Analytics for a Better World Webinars (ABW-W) [more information]

  1. Wednesday April 29, 2020,  EST 11AM (CET 17.00 PM), Speaker: Dimitris Bertsimas (MIT, Cambridge) | on a variety of aspects of COVID-19
  2. Wednesday May 27, 2020, CET 17.00 PM, Speakers: Koen Peters (World Food Programme and Zero Hunger Lab, Tilburg University), Hein Fleuren (Zero Hunger Lab, Tilburg University) | They will highlight Tilburg University's (The Netherlands) work at the United Nations - World Food Programme (WFP) headquarters in Rome

Discrete Optimization Talks (DOTs) [homepage]

Collection of Talks on COVID-19’s Impacts

INFORMS Chicago Chapter Webinar Series

  1. Modeling Supply Chain Decisions for Resource Sharing with an Application to Ventilator Allocation to Combat COVID-19 | Dr. Sanjay Mehrotra, Northwestern University | June 18, 2020 [paper] [info]
  2. Risk-based Thinking in a COVID-19 World | Dr. Sheldon Jacobson, University of Illinois | June 26, 2020 [info]

Managing Uncertainty: Adapting to and learning from the COVID-19 crisis - A Rotman Insights Webinar Series [link]

  1. Pandemic Protectionism: The Global Trade Impact of COVID-19, May 8, 2020
  2. Distribution Disruption: How to Build a Better Supply Chain after COVID-19, May 15, 2020

Articles

  1. Yuval Noah Harari: the world after coronavirus, March 20, 2020 | [financial times]
  2. Life in the Time of COVID18 - Prof Adam Przeworski, April 5, 2020 [Google Doc]

VMACS - Virtual Macro Seminar

  1. VMACS | Pandemics According to HANK - Greg Kaplan, Ben Moll, Gianluca Violante, March 31, 2020  | [YouTube] [slides]
  2. VMACS | Macroeconomic Implications of COVID-19: Can Negative Supply Shocks Cause Demand Shortages? April 2, 2020 | [YouTube]
  3. VMACS | Pandemics Depress the Economy, Public Health Interventions Do not: Evidence from the 1918 Flu | April 7, 2020 | [YouTube]
  4. VMACS | The Macroeconomics of Epidemics - Martin Eichenbaum, Sergio REbelo, and Mathias Trabandt, April 9, 2020 | [YouTube]
  5. VMACS | The Behavioral SIR Model, with Application to the Swine Flue Epedemic | For Whom the Bell Tolls: Avoidance Behavior at Breakout in COVID19, April 10, 2020 | [YouTube]

INFORMS online seminar

  1. From Pandemic Disruption to Global Supply Chain Recovery - Dr. Simchi-Levi, April 9, 2020 | INFORMS | [video] [slides]
  2. Ripple Effect in Supply Chains at Different Pandemic Stages - Dr. Dmitry Ivanov, May 14, 2020 | INFORMS
  3. Analytics in Turbulent Times - Alan Jacobson, June 17, 2020 | INFORMS [YouTube]
  4. Blood, Sweat, & PPEs: Rescuing Perishable Product Supply Chains & Impacting Policy through Analytics - Dr. Anna Nagurney, June 15, 2020 | INFORMS [YouTube]

Royal Economic Society Webinar Series

  1. Economic approaches for analysing the short, medium term and long run impact of the COVID-19 crisis -  Prof Tim Besley (Chair), Prof Daron Acemoglu, Prof Jean Tirole, April 9, 2020 | Royal Economic Society [video]

My Inspiration

 

(July 25, 2020)

J. D. Salinger, The Heart of a Broken Story

I think love is a touch and yet not a touch.

(May 25, 2020)

Romain Rolland

There is only one heroism in the world: to see the world as it is and to love it.

(Mar 21, 2020)

George Pólya

If you can’t solve a problem, then there is an easier problem you can solve: find it. -- How To Solve It: A New Aspect of Mathematical Method, G. Polya, Stanford University


(Feb 8, 2020)

Joseph Campbell

We must be willing to let go of the life we’ve planned so as to have the life that is waiting for us.


(Feb 6, 2020)

William Blake, English poet, painter, and printmaker. Largely unrecognized during his lifetime, Blake is now considered a seminal figure in the history of the poetry and visual arts of the Romantic Age. [more on Wikipedia]

What is now proved was once  only imagined.

I must create a system, or be enslaved by another man's. I will not reason and compare: my business is to create.

You never know what is enough unless you know what is more than enough.

Exuberance is beauty.

To generalize is to be an idiot.

That Man should labour and sorrow, and learn and forget and return to the dark valley whence he came, and begin his labours anew. -- The works of William Blake, poetic, symbolic, and critical


(Jan 19, 2020)

Voltaire, French philosopher:

Uncertainty is an uncomfortable position. But certainty is an absurd one.

Chinese proverb (according to A. Ben-Tal and L. El-Ghaoui. Robust Optimization. Princeton University Press, 2009; I haven't found the Chinese translation.):

To be uncertain is to be uncomfortable, but to be certain is to be ridiculous.


(Jan 18, 2020)

Friedrich Nietzsche:

He who has a why to live can bear almost any how.

 

HOWTO: Work with Small Datasets

(updated on January 12, 2020; not complete yet)

 

In general:

  1. 7 Effective Ways to Deal with a Small Dataset [link]
  2. Dealing with very small datasets [link]
  3. What to do with "small" data? [link]

 

 

For Images:

  1. Breaking the curse of small datasets in Machine Learning: Part 1 [link]
  2. Breaking the curse of small data sets in Machine Learning: Part 2 [link]
  3. You can probably use deep learning even if your data isn't that big [link]
  4. Applying deep learning to real-world problems [link]

HOWTO: Data Augmentation

[last updated on January 12, 2020; not complete yet]

 

Data Augmentation:

  1. Research Guide: Data Augmentation for Deep Learning, [Nearly] Everything you need to know in 2019 [link], keywords: Random Erasing Data Augmentation (2017), AutoAugment: Learning Augmentation Strategies from Data (CVPR 2019), Fast AutoAugment (2019), Learning Data Augmentation Strategies for Object Detection (2019), SpecAugment: for Automatic Speech Recognition (Interspeech 2019), EDA: for Boosting Performance on Text Classification Tasks (EMNLP-IJCNLP 2019), Unsupervised Data Augmentation for Consistency Training (2019)
  2. Data augmentation on entire dataset before splitting [link], conclusion: this practice is incorrect.
  3. How does data augmentation reduce overfitting? [link]

 

Data Augmentation for Regression Tasks:

Online articles that mentioned DA for regression tasks:

  1. Shehroz Khan's answer to What does the term data augmentation mean in the context of machine learning? [link]
  2. What you need to know about data augmentation for machine learning [link]
  3. Data augmentation techniques for general datasets? [link] (Teng: To me, it seems they were discussing feature engineering instead of adding more data points.)
  4. Data Augmentation Techniques for Cat/Binary/Continuous Numerical Dataset [link], keywords: SMOTE

 

 

Data Augmentation for Unbalanced Dataset in Classification Tasks:

  1. Oversampling and undersampling in data analysis [link]
  2. imbalanced-learn [GitHub] [docs]
  3. A collection of 85 minority oversampling techniques (SMOTE) for imbalanced learning with multi-class oversampling and model selection features [docs] [GitHub]
  4. A Deep Dive Into Imbalanced Data: Over-Sampling [link]
  5. SMOTE for high-dimensional class-imbalanced data, Rok Blagus and Lara Lusa, 2013 [link]
  6. SMOTE explained for noobs - Synthetic Minority Over-sampling TEchnique line by line [link]
  7. Detecting representative data and generating synthetic samples to improve learning accuracy with imbalanced data sets [link]
  8. ADASYN: Adaptive Synthetic Sampling Method for Imbalanced Data [link]

 

Data Augmentation for Image:

  1. Data Augmentation for Deep Learning [link], keywords: image augmentation packages, PyTorch framework
  2. 1000x Faster Data Augmentation [link], keywords: learn augmentation policies, Population Based Augmentation, Tune Framework
  3. A survey on Image Data Augmentation for Deep Learning, Connor Shorten and Taghi M. Khoshgoftaar [link]
  4. Python | Data Augmentation [link]
  5. How to Configure Image Data Augmentation in Keras [link]
  6. Data Augmentation | How to use Deep Learning when you have Limited Data -- Part 2 [link], keywords: online augmentation, offline augmentation
  7. Data augmentation for improving deep learning in image classification problem, Mikolajczyk et al. [link]
  8. The Effectiveness of Data Augmentation in Image Classification using Deep Learning, Jason Wang and Luis Perez [link]

 

Data Augmentation for Audio:

to be added ...

 

Data Augmentation for Texts:

  1. These are the Easiest Data Augmentation Techniques in Natural Language Processing you can think of -- and they work. Simple text editing techniques can make huge performance gains for small datasets. [link]

 

Data Augmentation for Time Series

  1. Data Augmentation strategies for Time Series Forecasting [link]

 

 

Academic Job Listings for Operations Management and Information Systems

OM/OR

  1. Operations Academia (link)
  2. Decision Science Institute Job Postings (link)
  3. POMS job openings (link)
  4. INFORMS Career Center (link) and mailing lists
  5. DMANET (link)
  6. ORNET(link)
  7. HigherEdJobs (link)
  8. AcademicJobsOnline (link)
  9. MathJobs (American Mathematical Society) (link)

IS

  1. Association for Information Systems Career Services (link) and mailing lists (link)

* By courtesy of Miao Bai, David Bergman, and Yuan Jin.

** Please let me know if I missed anything.