\chapter{Prediction-based Fleet Relocation}
\label{ch:method}

The state-of-the-art presented in Chapter~\ref{ch:background} shows that one of the main axes to improve a carsharing service is to relocate cars from areas with less usage to areas with a higher rate of customer usage.
The analysis of carsharing trips datasets in Chapter~\ref{ch:data_analysis} unveil that the three services have hot spots and cold spots, i.e. areas with respectively high and low demand.
On top of those spots, there are locations with a \textquote{deficit} or an \textquote{excess} of cars, meaning that relocation of cars in those cities is needed.
Since two services of the three available for study in this thesis have a fleet of electric vehicles, with different recharging strategies, the focus was made only on making a strategy for the relocation of misplaced vehicles.
That is, no discharged vehicle has to be taken into account and the operator should still manage manually the recharging of electric vehicles.
Thus, vehicles charged enough to be used by customers but placed in a low demand area is the focus of the relocation methodology presented as the main contribution of this thesis.

In this Chapter is presented an improvement of the methodology published as the main contribution of this thesis in the \emph{IEEE International Conference on Tools with Artificial Intelligence 2021}~\cite{martin_prediction_2021} and in the \emph{Conférence Nationale en Intelligence Artificielle 2022}~\cite{martin_optimisation_2022}.
This methodology is split in two parts that are detailed one after another.
The first part is about the daily customer usage prediction of a car.
That is the number of minutes the car is going to be spent rented by all the customers depending on the place where the car has been located in the morning at 6 a.m.
The second part is the automated decision of where cars should be placed.
It relies on the knowledge of where the cars were located before the \emph{relocation} phase is made, i.e the period when cars are being moved around by the staff.
It also relies on predictions of the customer daily usage for the next day.
Indeed in the case of the thesis, one of the constraints of the \emph{Free2Move} service is the inability to relocated cars during the day, i.e. vehicles of the service can be moved only during the night.

\section{Prediction of the Utility}

The first part of the methodology is the prediction of the daily usage of a car depending on which area it is placed.
As it should take into account that the cars can only be relocated at night, i.e. when a car is relocated during the night the staff won't be able to act on it until the next night, the objective is to predict the daily usage of the car, under the hypothesis that a car can be used multiple times during the day.
Placing the car at the right spot can enable it to make \textquote{chains of trips} during the day.
First is presented how the daily customer usage is modeled by the concept of \emph{Utility}.
Then the evaluation of different regression models is made, with the objective to find the type of model to use for the second part of the methodology.

\subsection{Modeling the Utility}

From the historical datasets of \emph{Free2Move}, a set of tuples to represent the user trips (\emph{trips}) has been created such that: 
$$\mathit{trips} = \{(\mathit{id}_{\mathit{vehicle}}, \mathit{id}_{\mathit{customer}}, \mathit{ts}_{\mathit{from}}, \mathit{ts}_{\mathit{to}}, \mathit{gps}_{\mathit{from}}, \mathit{gps}_{\mathit{to}}, \mathit{duration})\}$$
In these tuples, $\mathit{id}_{\mathit{vehicle}}$ is the unique ID of the vehicle used to make the trip.
The unique ID of the customer doing the trip is denoted $\mathit{id}_{\mathit{customer}}$.
The timestamp of departure and arrival of the trip are respectively referred as $\mathit{ts}_{\mathit{from}}$ and $\mathit{ts}_{\mathit{to}}$.
The information of the departure and arrival locations are respectively noted $\mathit{gps}_{\mathit{from}}$ and $\mathit{gps}_{\mathit{to}}$.
They are provided as GPS coordinates, which can be mapped to the geographical cells defined earlier.
From the timestamps (e.g. $\mathit{ts}_{\mathit{from}}$) additional features can be derived, such as whether the day is a workday or a holiday, the day of the week or weather information.
The number of minutes driven ($\mathit{duration}$) is an additional trip information that is deduced from the difference between $\mathit{ts}_{\mathit{from}}$ and $\mathit{ts}_{\mathit{to}}$.
Trips made by the staff, i.e. trips by employees with the purpose of relocating vehicles overnight, are excluded from the data.

\paragraph{Actual utility.} Based on the historical data of a single day, the total utility $U_k(n)$ is computed for each cell $k$ given that there were $n$ cars available in that cell at the start of the day.
It is insufficient to only count the utility of the trips from this cell to other cells, as the cars may have been used for subsequent trips.
Based on this observation, the total utility of each vehicle is computed as the total time the vehicle has been rented during the day: 
$$U(\mathit{id}_{\mathit{vehicle}}) = \sum_{t \in \mathit{trips}(\mathit{id}_{\mathit{vehicle}})} \mathit{utility}(t)$$
where $\mathit{trips}(\mathit{id}_{\mathit{vehicle}})$ is the set of trips that have this vehicle ID and the utility $\mathit{utility}(t)$ of a trip $t$ is its duration.

Let $\mathit{initial\_vehicles}(k)$ be the set of vehicle identifiers that were in a cell $k$ at the start of the day, with $|\mathit{initial\_vehicles}(k)|=n$.
Then the \emph{actual utility} of a cell $k$ which had $n$ initial vehicles is measured as: 
$$U_k(n) = \sum_{\mathit{id}_{\mathit{vehicle}} \in \mathit{initial\_vehicles}(k)} U(\mathit{id}_{\mathit{vehicle}})$$ 
Note that this value can only be computed if $n = |\mathit{initial\_vehicles}(k)|$, i.e for a given day in a cell $k$ with $\mathit{initial\_vehicles}(k)$ it is not possible to know $U_k(x)$ if $x \neq |\mathit{initial\_vehicles}(k)|$.

\begin{figure}[!p]
    \centering

    \includegraphics[width=0.8\linewidth]{figure/ch3_method_summary.jpeg}

    \caption[Graphical summary of the method]{Overview of the approach: a regression model is trained for each cell to predict, from a historical dataset, the utility of each car (by order of decreasing utility), knowing both time and weather-related features. The optimization solver uses both the set of models learned to predict the utility of each car and the night car positions to return the best next morning car positions.}
    \label{fig:method_summary}
\end{figure}

\paragraph{Estimated utility.} The goal now is to predict $U_k(n)$ for any cell $k$ and value $n$, without knowing which trips will be demanded and which vehicles will be used for them.
This can be cast as a regression problem, where the training data are the actual utilities defined above.
However, only the $U_k(n)$ for one $n$ per cell is observed per day, this makes a very sparse training set, unlikely to lead to a good regression model for arbitrary values of $n$.

Thus the regression problem of $U_k(n)$ is decomposed, in order to have more training data and more refined estimates. 
First the cars of a cell are ordered in a deterministic way, namely by daily decreasing utilization order (most used car in 1$^{st}$, second most used car in $2^{nd}$, etc.). 
The rank of a car in this ordered list is denoted $i$, such that the 1$^{st}$ car in the order has a rank $i=1$.
The gain made by adding a $i^{th}$ vehicle in the area can now be defined as $U^i_k = \mathit{utility}(\mathit{id}_{\mathit{vehicle}})$ with $id_{vehicle}$ the initial vehicle in cell $k$ with its rank $i$ as its rank in the ordered car list.
It is done such that: $U_k(n) = \sum_{i \in 1..n} U^i_k$. 
Based on this reformulation, the goal of predicting the \emph{gain of utility} $U^i_k$ brought by the $i$th car in a region $k$ is identified. 
This decomposition of the utility value and the more fine-grained regression problem allows to have more training data, and thus to better estimate $U_k(n)$ for different values of $n$ (especially for $n < |initial\_vehicles(k)|$).

\paragraph{Prediction}
The objective is to predict $U^i_k$ for each cell $k$ and rank $i$ using the previously defined training labels, this prediction is called $\hat{U}^i_k$. 
As the cars utilization in each cell has different behavior, one function $f_k$ is learned per cell.
As features, the rank $i$ is used as well as a set $\mathit{fs}$ of features including weather features, temporal features and lagged observations of the previous days (as the data is sequential). 
Hence, the aim is to learn a regression function $f_k$ per cell such that: 
$$\hat{U}^i_k = f_k(i,\mathit{fs}).$$
Regression methods to use are discussed in Section~\ref{sec:utility_experiment_regressors}.

\paragraph{Clustering cells}
Instead of learning one predictor $f_k$ per cell, similarities in utility behavior for different cells can also be used.
By grouping cells, more data becomes available for every function that are going to be learned, and ideally more accurate predictions are made.
Hence investigations are made on the effect of clustering the cells which behave similarly in terms of their car utility, and learn one function $f_c$ per cluster $c$.
The two clustering approaches used during the experiments are detailed in Section~\ref{sec:utility_experiment_clustering}.

\subsection{Utility Prediction Evaluation}
\label{sec:utility_experiment}

The evaluation of the prediction of the utility of cars is done for the three services detailed in Chapter~\ref{ch:data_analysis}, that is \emph{Madrid}, \emph{Paris} and \emph{Washington}.
The aim of this evaluation is to assess which kind of regression model and which parameters should be used for the second part of the methodology to be detailed in Section~\ref{sec:ch3_fleet_relocation}.
First is defined the three cell-clustering strategies that are going to be used during the experiment.
Then the regression models to be used during the following experiments are detailed.
In the third part is detailed the metrics that are going to measure the performance of the regression models.
Finally, the case study on the three cities is exposed.

\subsubsection{Cell clustering}
\label{sec:utility_experiment_clustering}

For this case study, three cell clustering strategies are compared.
A clustering strategy is the grouping of cells together to train a single regression model for all the cells of each cluster, instead of learning one regression model per cell.
The first strategy is a baseline strategy where no clustering is done at all (\emph{No Cluster}).
In this strategy, each function $f_k$ is modeled independently for all the cells $k$ of the city grid.

The second clustering strategy is based on a \emph{K-Means}~\cite{least_1982_lloyd} clustering where each cell is represented by a \emph{vector}.
This vector is created such that the $i^{th}$ coordinate of the vector is the average utility (in minutes) of the car of rank $i$.
The optimal number (for prediction purpose) of clusters is found using the validation set for each model and feature set.
The utility of a cell $k$ becomes $U^i_k = f_c(i, \mathit{fs})$ where $f_c$ is the predicted utility of the cluster $c$ which contains the cell $k$.
To compute the K-means distances, all vectors have the same fixed size which is the highest car ranked $i$ encountered in the dataset.
This size is set to $i=20$ for the three datasets.
If a cell has no average driven time for a rank $i$, the $i^{th}$ coordinate is filled with a $0$ in the corresponding vector.
In the experiments, the Scikit-learn implementation~\cite{scikit_learn} of K-Means is used with the Euclidean distance.

The third clustering strategy is based on the same \emph{vectors} of utility.
However, this approach with a \emph{K-Medoids} uses a different distance computation strategy.
First, the size of the vector describing each cell is the highest car rank encountered in this particular cell.
Thus all the cell vectors do not have the same size. 
The distance between two such vectors of different sizes $i$ and $i+h$ (with $h>=0$) is only computed on the first $i$ common car ranks between both vectors.
It is the Euclidean distance between these two common parts divided by its length $i$.
In the experiments, the Pyclustering~\cite{Novikov2019} implementation of the K-Medoids algorithm is used.


\subsubsection{Utility Prediction Algorithms}
\label{sec:utility_experiment_regressors}

The performance of two baselines and three regression models is evaluated in this section: two baselines consisting of a \emph{Zero} predictor, the \emph{Mean} of historical values, \emph{Gradient Boosting Decision Tree} regressors (\emph{GBR})~\cite{GBT}, \emph{Support Vector Machine} regressors (\emph{SVR})~\cite{SVR} and linear \emph{ElasticNet} regressors~\cite{zou2005regularization} are considered.
Two sets of features (\emph{Rank} and \emph{All}) were used to train the models.
Other machine learning algorithms and learning strategies were tested but the results were unconvincing and are not presented here. 

\paragraph{Features} The first set of features \emph{Rank} contains no exogenous features, such that only the rank $i$ of each car is used to train all the regression models. 
The second set of features \emph{All} contains all the exogenous information available: the \emph{day of week}, whether this day is a \emph{workday or not}, \emph{weather information} and two \emph{historical features}.
As detailed in Chapter~\ref{ch:data_analysis}, the weather information is obtained by pre-processing the hourly weather broadcasts data.
To fit the daily prediction, an average view of the weather for each day has been created.
For each day, one weather vector is then created containing the normalized values of the computed hourly averages (temperature, relative humidity, pressure, wind speed, cloud cover, rain and snow).
Then these vectors are used in a clustering model to create $4$ different weather clusters, with the idea to associate utilization to general weather conditions rather than each individual weather information.
For the regression methods, the cluster value is used as a feature in a one-hot encoding fashion. 
Then, the two historical features are given for each cell $k$ with a given rank $i$: the last two (in time) utility values $f_k(i)$ known (not predicted) in the data.

\paragraph{Models} The first baseline (\emph{Zero}) is returning the value \textquote{0} when the prediction of the utility of a car is asked.
This baseline predictor draws the line that should not be crossed by a predictor, i.e. a predictor whose performance is worse than always predicting a null value is not of interest.
The second baseline model (\emph{Mean}) is the average of the utility values computed for each cell, for a given rank $i$ for all days in the training set.
It is the equivalent of what is used in the previous works presented in Chapter~\ref{ch:background}, that is taking into account only the average past customer utilization.
These two baselines only use the feature set \emph{Rank}. 

In the following experiment is used the regression variants of Scikit-learn implementations of the Gradient Boosting Tree (\emph{GBR}), of the Support Vector Machine (\emph{SVR}) and of the \emph{ElasticNet} with their default hyperparameters, other values have been tested without significant performance change.
Each model directly predicts the utility of the car in its corresponding (group of) cell(s) for the two previously mentioned feature sets.
Each attribute had been normalized by subtracting the average and dividing by the standard deviation.
Experiments were not done with other well-known machine learning models (such as Neural Networks or Random Forests) because of the small number of attributes available to describe the data, the small number of utility entries available and the presence of the mandatory \emph{car rank} attribute.
Mandatory attributes are not suitable for the Random Forests algorithm.

Other machine learning algorithms and learning strategies were tested but gave unconvincing results. 
This includes an alternative two-step approach: first a classification model was used to predict whether a car leaves the cell or not and, in case it does leave, a second model is used to predict the utility of this car.
Since the first classification model was far from perfect (around 70\% accurate), it propagated a lot of errors in the second step and the whole two-step process obtained worse results than previously proposed current predicting strategy.
The second approach tested did not use $i$ as a feature and directly predicted the whole vector of utility for each cell, i.e it used multi-target regression models. 
These approaches both performed worse than the baseline model \emph{Mean}, and hence they are not reported in the following.
It should also be noted that when a type of model is chosen, then all the (group of) cells has the same type of regressor to predict the utility.
An approach selecting first the type of model to train for each cell, including \emph{Zero} and \emph{Mean}, has been tested but shown no better result than in the best case with all the (group of) cells using the same type of model.

\subsubsection{Evaluation Metrics}

The regression model performance is assessed through two different metrics, an absolute and a relative one.
The first one, a \emph{Mean Absolute Error}~(MAE) is the average number of minutes either overestimated or underestimated by the trained regression models on each area for one day.
In the following case study, the daily average of the MAE is computed such that the values reported in the tables are calculated by :

$$ \mathit{MAE} = \frac{1}{|T|} \cdot \frac{1}{|K|}  \sum_{t \in T} \sum_{k \in K} \sum_{i \in I} |obs_{tki} - pred_{tki}| $$

With $T$ the set of timestamps in the data, $obs_{tki}$ the real utility of the i$^{th}$ car in cell $k$ at timestamp $t$ and $pred_{tki}$ the predicted utility of the i$^{th}$ car in cell $k$ at timestamp $t$.

Then as a relative measure to better understand the regression model performances, a ratio between the MAE and the average real driven time per cell of the same day is also used, its is called here the \emph{Ratio Mean Absolute Error}~(RMAE).
As for the previous metric, the daily average of its values is reported such that the values are computed by : 

$$\mathrm{RMAE} = \frac{1}{|T|} \cdot \sum_{t \in T} \frac{\frac{1}{|K|} \sum_{k \in K} \sum_{i \in I} |obs_{tki} - pred_{tki}|}{\frac{1}{|K|} \sum_{k \in K} \sum_{i \in I} obs_{tki}} $$

\subsubsection{Case Studies}

The evaluation of the utility prediction is done for the three services that are presented in Chapter~\ref{ch:background}.
The objective is to evaluate how well can $f_k(i, \mathit{fs})$ be predicted for a cell $k$, a car rank $i$ and an exogenous feature set $\mathit{fs}$.
This is done for the three regressions approaches (\emph{ElascticNet}, \emph{GBR} and \emph{SVR}), with their comparison to the two baselines (\emph{Zero} and \emph{Mean}) described earlier.

\begin{table*}[!tbh]
    \hspace*{-1.75em}
    % \footnotesize
    \small
    \centering
    \begin{tabular}{|l|l|m{1.8em} m{2.8em}|m{1.8em} m{2.8em}|m{1.8em} m{2.8em}|m{1.8em} m{2.8em}|m{1.8em} m{2.8em}|}
    \hline
    \multirow{3}{*}{Clustering} & \multirow{3}{*}{Features} & \multicolumn{10}{l|}{Madrid} \\ \cline{3-12} 
        &  & \multicolumn{2}{l|}{Zero} & \multicolumn{2}{l|}{Mean} & \multicolumn{2}{l|}{ElasticNet} & \multicolumn{2}{l|}{GBR} & \multicolumn{2}{l|}{SVR} \\ \cline{3-12}
        &  & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE \\ \hline
    \multirow{2}{*}{No Cluster} & Rank & 775  & 100\% & 263 & 36\% & 276 & 38\% & 263 & 36\% & 273 & 37\% \\ %\cline{2-6} 
        & All & N/A & N/A & N/A & N/A & 247 & 34\% & 245 & 33\% & 288 & 39\% \\ \hline
    \multirow{2}{*}{KMeans} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 266 & 36\% & 280 & 38\% & 265 & 36\% & 266 & 36\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 250 & 34\% & 240 & 33\% & 247 & 34\% \\ \hline
    \multirow{2}{*}{KMedoids} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 266 & 36\% & 280 & 38\% & 266 & 36\% & 266 & 36\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 249 & 34\% & \textbf{239} & \textbf{33\%} & 243 & 33\% \\ \hline
    \hline
    \multirow{3}{*}{Clustering} & \multirow{3}{*}{Features} & \multicolumn{10}{l|}{Paris} \\ \cline{3-12} 
        &  & \multicolumn{2}{l|}{Zero} & \multicolumn{2}{l|}{Mean} & \multicolumn{2}{l|}{ElasticNet} & \multicolumn{2}{l|}{GBR} & \multicolumn{2}{l|}{SVR} \\ \cline{3-12}
        &  & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE \\ \hline
    \multirow{2}{*}{No Cluster} & Rank & 85 & 100\% & 69 & 86\% & 83 & 104\% & 64 & 77\% & 66 & 79\% \\ %\cline{2-6} 
        & All & N/A & N/A & N/A & N/A & 75 & 92\% & 64 & 76\% & 73 & 86\% \\ \hline
    \multirow{2}{*}{KMeans} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 69 & 86\% & 85 & 107\% & 64 & 77\% & 64 & 77\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 77 & 94\% & \textbf{62} & \textbf{74\%} & 67 & 79\% \\ \hline
    \multirow{2}{*}{KMedoids} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 71 & 89\% & 87 & 110\% & 65 & 78\% & 65 & 78\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 78 & 96\% & 64 & 76\% & 67 & 79\% \\ \hline
        \hline
    \multirow{3}{*}{Clustering} & \multirow{3}{*}{Features} & \multicolumn{10}{l|}{Washington} \\ \cline{3-12} 
        &  & \multicolumn{2}{l|}{Zero} & \multicolumn{2}{l|}{Mean} & \multicolumn{2}{l|}{ElasticNet} & \multicolumn{2}{l|}{GBR} & \multicolumn{2}{l|}{SVR} \\ \cline{3-12}
        &  & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE & MAE & RMAE \\ \hline
    \multirow{2}{*}{No Cluster} & Rank & 198 & 100\% & 226 & 121\% & 252 & 136\% & \textbf{173} & \textbf{87\%} & 176 & 88\% \\ %\cline{2-6} 
        & All & N/A & N/A & N/A & N/A & 250 & 132\% & 174 & 88\% & 187 & 94\% \\ \hline
    \multirow{2}{*}{KMeans} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 227 & 122\% & 257 & 138\% & 175 & 88\% & 175 & 88\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 249 & 130\% & \textbf{173} & \textbf{87\%} & 180 & 91\% \\ \hline
    \multirow{2}{*}{KMedoids} & Rank & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & 233 & 125\% & 262 & 141\% & 181 & 92\% & 181 & 92\% \\ %\cline{2-6} 
        & All & & & N/A & N/A & 251 & 131\% & 176 & 89\% & 180 & 91\% \\ \hline
    \end{tabular}
    \caption{MAE (lower is better) and RMAE performances (lower is better) of the car utility prediction per day and per (group of) cell(s). One table is used to described the results for one service. Each time, two baselines (\emph{Zero} and \emph{Mean}) and three regression models (\emph{ElasticNet}, \emph{GBR} and \emph{SVR}) with two sets of features (\emph{Rank} and \emph{All}) are tested.}
    \label{tab:ch3_utility_regression}
\end{table*}

The columns of Table~\ref{tab:ch3_utility_regression} represent the regression models used, i.e the \emph{Zero} and historical \emph{Mean}, the \emph{ElasticNet}, the \emph{Gradient Boosting Tree} (\emph{GBR}) and the \emph{Support Vector Machine} (\emph{SVR}).
The lines describe the results with or without clustering using the feature sets presented before. 
MAE (in min) and RMAE (in \%) are related to the true number of minutes driven in each cell and day and the predicted number of minutes on the validation set.
The performance of the baseline \emph{Zero} has been computed only once since its output does not depend on any feature, and would return the same results for each configuration.
The historical \emph{Mean} performance has not been computed while using the feature set \emph{All} since its statistical value could be questionable.
It should be noted that for all the combinations of settings, the training and evaluation of each regression models took less than 5 minutes.
Each combination is thus compatible for a daily usage by an operational team in each service.

\paragraph{Madrid.} 

One can notice that with the feature set \emph{Rank}, the linear regression model gives one of the worst MAE on average without clustering ($276$ mins) and with any kind of clustering approach ($280$ mins).
This shows the potential of more complex machine learning models and of our chosen exogenous features when there is enough data.

Indeed, even if the models \emph{GBR} or \emph{SVR} do not perform better than the historical \emph{Mean} with only the feature set \emph{Rank}, the addition of the exogenous features from the feature set \emph{All} improves the prediction performance in most of the cases and leads to beat the historical \emph{Mean}.
This improvement is augmented by regrouping cells together with the help of any kind of clustering approach, with \emph{K-Medoids} being slightly better than \emph{K-Means} for \emph{Madrid}.

Globally it can be noticed that compared to the other services, the regression models make a higher absolute error in the case of \emph{Madrid}.
However the historical utilization of this service is higher, with on average $775$ minutes of daily historical utility per cell.
This high rate of absolute error should be mitigated by the relative measure (\emph{RMAE} in \%) indicating that this absolute error represents on average each day only $\sim$33\% of the average historical utility in the best case. 

These results show that the configuration to choose for the regression approach is the \emph{GBR} regression model with a \emph{K-Medoids} clustering of the similar cells while using the feature set \emph{All}.
This combination offers a $3\%$ reduction in RMAE compared to the baseline \emph{Mean}, as well as a reduction of $24$ minutes (on average per cell and per day) in the MAE.
This setting will be used for \emph{Madrid} in the following experiments.

\paragraph{Paris.}

This dataset brings a bigger challenge to the prediction algorithm than the first one. 
The first reason is that the number of cars is lower in this service ($475$ compared to $578$ in \emph{Madrid}) while the number of cells covered by the service is slightly larger ($209$ compared to $155$ in \emph{Madrid}), thus the spatial density of cars is lower.
Moreover the global utilization of the service itself is lower than for \emph{Madrid}, with a daily historical utilization on average of $85$ minutes per cell, with a utility for each car often (at a rate of $55\%$) being null in the dataset.

This could explain the relatively bad performance of the historical \emph{Mean} baseline ($\sim$69 min of MAE which corresponds to almost $86\%$ of the average per day and cell of the historical minutes driven) compared to the prediction of the machine learning models in Table~\ref{tab:ch3_utility_regression}.
For this dataset, the \emph{GBR} algorithm performs better than \emph{SVR}. As for the first dataset, the results are better when clustering the cells except for the \emph{SVR} model.

It can be noted that with or without clustering and with the \emph{Rank} feature set, the linear regression model \emph{ElasticNet} performs badly with a relative error superior to $100\%$.
For those settings, \emph{ElascticNet} is worse than using the model \emph{Zero} which only predict $0$ for each utility, which would have on average a MAE of $86$ minutes per day and per cell and on average a RMAE of $100\%$ per day.

The best prediction results provide a MAE of $\sim$62 minutes which correspond to $74\%$ of the average per day and cell of the total minutes driven for the setting using the \emph{K-Means} clustering approach with the feature set \emph{All} and a \emph{GBR} regression model.
As it will be detailed in the next section, these far from perfect prediction results, allow us to improve the overall service anyway which gives hope for a much larger margin of improvement.

\paragraph{Washington.}

As for \emph{Paris}, this dataset brings a bigger challenge for the regression models too.
Indeed while the number of cars is around the same as in \emph{Madrid} with $600$ cars, the number of cells covered by the service is twice as much, with $411$ cells for \emph{Washington} against $155$ cells for \emph{Madrid}. 
Thus the density of cars is halved compared to the first dataset.
Besides, while on average the utility in each cell ($198$ minutes) is higher than for \emph{Paris}, it is explained by the facts that each individual trip is on average longer than those in the \emph{Paris} and that on average half of the cars have a daily utility of zero.
Thus the regression models need to take into account a higher variance of utilities in the dataset.

As for \emph{Paris}, this explains the bad performance of the \emph{Mean} baseline which has on average has a RMAE of $121\%$ in the best setting, with \emph{No Cluster} and the feature set \emph{Rank}.
As for the two previous datasets, the linear regression model \emph{ElasticNet} is not capable of modeling the utility better than the baseline.
It means that using the baseline \emph{Mean} to predict the values of the utility for the cars is worse than simply predicting zeros.
However both the models \emph{GBR} a \emph{SVR} perform better than the baseline and with any setting stays under a RMAE of $100\%$.

Even if the utility is difficult to model in the case of \emph{Washington}, for the following experiments we select the setting using the \emph{K-Means} clustering approach with the feature set \emph{All} and the regression model \emph{GBR}.
This setting performs the best with a MAE of $173$ minutes, reducing the MAE by $53$ minutes ($34$ \%) compared to the \emph{Mean} baseline.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% SECTION - SECTION - SECTION - SECTION - SECTION - SECTION - SECTION - SECTION - SECTION - SECTION - SECTION %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Fleet Relocation}
\label{sec:ch3_fleet_relocation}

The second part of the methodology to relocate the vehicles is the optimization of the placement of the fleet.
It should take into account the main constraint that the number of relocation possible is limited by the number of employees, also called jockeys, and the time and cost needed to do each relocation.
Additionally, this means that the previous placement of the vehicles has to be taken into account too.
This is done by using an \emph{Integer Linear Programming} formulation that is given to a solver in order to find the best placement of the cars.
In order to design this optimization formulation, a first and simpler proposition has been formulated on the optimization of the vehicle position.
Then its improvement to be able to take into account the jockey capabilities and the previous fleet status is detailed.
A greedy algorithm to solve the formulation, and to act as a baseline, is also exposed.
Lastly the evaluation, in terms of revenues, of the impact of the fleet relocation is done on the historical dataset.
The aim is to compare the proposed method, the thesis contribution, against a greedy baseline and the historical revenues.

\subsection{Naive Relocation}

As a first approach, the car relocation problem boils down to determining the best placement of the fleet over the different areas.
Indeed given $\hat{U}^i_k$, the predicted utilization of the $i$th car located in the cell $k$, and a bound $\Gamma$ on the number of cars available, 
the determination of the best placed of the $\Gamma$ cars over the different cells can be formulated.
Note that if $\hat{U}^i_k$ would be linear in $i$, e.g. $\hat{U}^i_k = i*w$ for some value $w$, then the relocation problem would be an unbounded knapsack problem with capacity $\Gamma$, where each cell is one item.
However, since this simple linearity assumption does not hold, a different formulation is used to model the relocation problem.
More specifically, an \emph{Integer Linear Programming} (\emph{ILP}) is used to model it as an assignment problem of vehicles to cells, where the total capacity $\Gamma$ and the order of the vehicles in each cell must be respected. 

Boolean indicator variables $V_{ki}$ are used to represent whether there should be an $i$th vehicle in cell $k$ in the next morning.
The total number of vehicles in a cell $k$ is hence $\sum_i V_{ki}$.
Let $K$ be the set of all cell identifiers defined earlier.
Let $m$ be an upper bound on the number of vehicles allowed in one cell and $I=\{1..m\}$.
Let $V=(V_{ki})_{k \in K, i \in I}$ be the matrix of indicator variables whose value is searched with a solver.
The \emph{ILP} formulation of the problem is then:

\begin{align}
    \underset{V}{maximize}\quad &\sum_{k \in K} \sum_{i \in I} \hat{U}_k^i \cdot V_{ki} \label{eq:objectiveNoRelocCost}\\
          s.t.\quad &\sum_{k \in K} \sum_{i \in I} V_{ki} \leq \Gamma \label{eq:constraintSimpleCapacity}\\
                    &V_{ki} \geq V_{ki+1} \quad &&\forall k \in K, i \in I\setminus\{m\} \label{eq:constraintSimpleGap} \\
                    &V_{ki} \in \{0,1\} &&\forall k \in K, i \in I
\end{align}

The objective function of this problem is Equation~\ref{eq:objectiveNoRelocCost}, stating that the values of $V$ are searched to maximize the sum of the utility of the placed vehicles.
If the indicator variable $V_{ki}$ is equal to $1$ than the value of $\hat{U}_k^i$ is taken into account in the sum, else it is not since no vehicle would be placed in cell $k$ at rank $i$.
Equation~\eqref{eq:constraintSimpleCapacity} is the capacity constraint, meaning that no more than $\Gamma$ cars can be placed in the city.
Equation~\eqref{eq:constraintSimpleGap} ensures that there is no `gap' in the rank of the indicator variables, that is, if $V_{ki} = 0$ then any $j > i$ has to be $0$ too.
For example, if there is no 3rd vehicle in a cell, there is also no 4th, 5th, etc. vehicle.

\subsection{Jockey-based Relocation}
\label{sec:ch3_jockey_relocation}

The above formulation computes the optimal placement but does not consider the fact that 1) at the end of the day, the vehicles are already located in certain cells, and 2) cars need to be relocated by jockeys, raising the need to take into account their limited capacity and the associated costs of each relocation.

To address 1), $s_{ki} \in \{0,1\}$ denotes whether there is an $i$th vehicle in cell $k$ at the end of the previous day.
To address 2), a limit $\gamma$ is included to limit the total number of possible car relocations that the jockeys can do.
The number of ranks that are used in the formulation, the upper limit of $I$ called $m$, is redefined as $m = \mathit{max}(n, \nu)$, with $n$ the maximum number of cars observed in a single cell before relocation and $\nu$ a chosen upper limit to the number of cars to be placed inside each cell after relocation.
This is done to take into account both a limit in the number of cars to be placed in each area $\nu$ and the maximum number of cars that are actually present in all cells before relocation $n$.
Furthermore jockeys cannot teleport; hence they must be moved by a so-called `sweeper' car to the next car after dropping off a relocated vehicle.
Operationally these sweeper trips should be minimized, $q_{kl}$ is introduced as the cost of a jockey being moved by a sweeper car from cell $k$ to cell $l$.
A price $p$ per minute paid by users renting a car is also defined so that revenues from cars being rented and costs from the relocation can be balanced off.

The goal now is to find the relocation that leads to the largest possible expected revenue minus the associated jockey sweeping costs, while being to perform at most $\gamma$ relocations.
To model this as an \emph{Integer Linear Programming} problem, two sets of Boolean variables are used: $F_{ki}$ denotes that the $i$th car is relocated away \emph{from} cell $k$, while $T_{ki}$ denotes that a car is relocated \emph{to} cell $k$ to be its $i$th car. 
Using these binary variables, whether or not there is a car at the $i$th position in cell $k$ is computed by $s_{ki} - F_{ki} + T_{ki}$, which replaces $V_{ki}$ from the previous formulation.
Finally, $E_{kl}$ represents the number of sweeper trips necessary between cell $k$ and $l$, with $q_{kl}$ the cost of one such sweeper trip.

Plugging that into the above optimization problem and adding appropriate extra constraints leads to:

\begin{align}
    \underset{F,T,E}{argmax}\quad &\sum_{k \in K} \sum_{i \in I} p \cdot \hat{U}_{ki} \cdot (s_{ki} - F_{ki} + T_{ki}) - \sum_{k \in K} \sum_{l \in K}  q_{kl} \cdot E_{kl}  \label{eq:relocObjective}\\
    % minimize_{E}\quad &\sum_{k \in K} \sum_{i \in I} \sum_{l \in K} \sum_{j \in I} E_{kilj} \cdot d_{kl} \label{eq:minDistance}\\
          s.t.\quad &s_{ki} - F_{ki} \geq 0 &&\hspace{-3.5em} \forall k \in K, i \in I \label{eq:relocDontRemoveNone}\\ % don't remove if not present
                    &s_{ki} + T_{ki} \leq 1 &&\hspace{-3.5em} \forall k \in K, i \in I \label{eq:relocDontAddTooMany}\\ % don't add if already present
                    &\sum_{k \in K} \sum_{i \in I} T_{ki} = \sum_{k \in K} \sum_{i \in I} F_{ki} \label{eq:relocAddSameRemove}\\
                    &\sum_{k \in K} \sum_{i \in I} T_{ki} \leq \gamma, \sum_{k \in K} \sum_{i \in I} F_{ki} \leq \gamma \label{eq:relocLimitBudget}\\
                    &(s_{ki} - F_{ki} + T_{ki}) \geq (s_{ki+1} - F_{ki+1} + T_{ki+1}) &&\hspace{-3.5em} \forall k \in K, i \in I\setminus\{m\} \label{eq:relocGap} \\
                    &\sum_{i \in I} (s_{ki} - F_{ki} + T_{ki}) \leq \nu &&\hspace{-3.5em} \forall k \in K \label{eq:maxCarPerCell} \\
                    &\sum_{l \in K} E_{kl} = \sum_{j \in I} T_{kj}&&\hspace{-3.5em} \forall k \in K \label{eq:outgoingJockey}\\
                    &\sum_{k \in K} E_{kl} = \sum_{i \in I} F_{li} &&\hspace{-3.5em} \forall l \in K \label{eq:incomingJockey}\\
                    &F_{ki} \in \{0,1\}, T_{ki} \in \{0,1\} &&\hspace{-3.5em} \forall k \in K, i \in I \\
                    &E_{kl} \in \{0, \dots, |I|\} &&\hspace{-3.5em} \forall k, l \in K
\end{align}

The objective function in Equation~\eqref{eq:relocObjective} maximizes the total expected revenue from the utility over the computed car placement while taking into account the costs to reposition jockey between cars to relocate.
Equation~\eqref{eq:relocDontRemoveNone} states that only vehicles that are present at the cell and rank coordinate at the end of the day can be moved away overnight, while Equation~\eqref{eq:relocDontAddTooMany} states that if there is already a vehicle present at this cell and rank coordinate then another vehicle cannot be moved to the location.
These two constraints effectively state that depending on $s_{ki}$ either $F_{ki}$ or $T_{ki}$ has to be 0. 
Equation~\eqref{eq:relocAddSameRemove} states that the number of pickups and drop-offs has to be the same, while Equation~\eqref{eq:relocLimitBudget} ensures that both the number of pickups and drop-offs is lower than the relocation budget $\gamma$.
Equation~\eqref{eq:relocGap} ensures that there are no ``gap'' in the final assignment, obtained by substituting $s_{ki} - F_{ki} + T_{ki}$ into Equation~\eqref{eq:constraintSimpleGap} from the previous formulation.
The constraint proposed in Equation~\eqref{eq:maxCarPerCell} makes sure that it is not possible to have more than $\nu$ cars in each cell after the relocation process.
However it should be noted that it may make the problem infeasible with low values for $\gamma$ (provided as a constant beforehand), i.e too many cars can be already present above the $\nu$ limit and there may not be enough relocations available to move they away.
Finally Equation~\eqref{eq:outgoingJockey} makes sure that each time a car is relocated \emph{to} $k$, then a sweeper car needs to pick up the jockey.
Equation~\eqref{eq:incomingJockey} assures that there is a sweeper car dropping of a jockey for each car relocated \emph{from} $l$.

During the experiments, in Section~\ref{sec:ch3_relocation_evaluation} a variant of this formulation is explored, it consists in replacing the objective function with Equation~\eqref{eq:relocObjectiveVariant1} and Equation~\eqref{eq:relocObjectiveVariant2}.
In this variant, $F$ and $T$ are first searched to maximize Equation~\eqref{eq:relocObjectiveVariant1}, then without changing the values of $F$ and $T$ the values of $E$ are searched to minimize Equation~\eqref{eq:relocObjectiveVariant2}:

\begin{align}
    \underset{F,T}{argmax} &\sum_{k \in K} \sum_{i \in I} p \cdot \hat{U}_{ki} \cdot (s_{ki} - F_{ki} + T_{ki}) \label{eq:relocObjectiveVariant1}\\
    \text{then }\underset{E}{argmin} &\sum_{k \in K} \sum_{l \in K}  q_{kl} \cdot E_{kl} \label{eq:relocObjectiveVariant2}
\end{align}


\subsection{Greedy Relocation}
\label{sec:ch3_greed_relocation}

The \emph{Integer Linear Programming} approach proposed above is compared against a baseline to assess the gain made by it.
For this baseline, a greedy algorithm approach has been adopted and propose a resolution to guarantee that the constraints previously presented \emph{Integer Linear Programming} are respected too.

The greedy approach takes the same input as the formulation described previously in Section~\ref{sec:ch3_jockey_relocation} and produces the same kind of output as expected from this formulation.
For the inputs, the value of $s_{ki}$ for all cells $k$ and ranks $i$ are given in a matrix $s$.
The utility $U^i_k$ of all combinations of cell $k$ and rank $i$ are in the matrix $U$.
Two constants $\nu$ and $\gamma$ are similar to the ones described in the previous formulation, i.e. the first represent the maximum number of cars that should be present in each cell and the second the maximum number of relocation that can be made.
The greedy algorithm outputs three matrices similar to the matrices of variables returned by the previous formulation.
The matrix $F$ represents whether or not a car is taken from the cell $k$ at rank $i$ for all cells and ranks possible.
The matrix $T$ represents whether or not a car is added to the cell $k$ at rank $i$.
The matrix $E$ denotes the number of jockey being moved by a sweeper car between every combination of two cells. 
A summary of the inputs required by the greedy algorithm and its output is described in Table~\ref{emphtab:ch3_summary_io_greedy}.

\begin{table}[bth]
    \centering
    \small
        \begin{tabular}{c p{300px}}
        \hline
        Input Name & Description \\ \hline
        $s$ & Matrix ($|K|$ rows, $|I|$ columns), placement of cars before relocation. \\
        $U$ & Matrix ($|K|$ rows, $|I|$ columns), predicted utility of all cells and rank combinations. \\
        $\nu$ & Maximum number of cars per cell after relocation. \\
        $\gamma$ & Number of authorized relocation \\ \hline
        \hline
        Output Name & Description \\ \hline
        $F$ & Matrix ($|K|$ rows, $|I|$ columns), car removal indicator in cell $k$ rank $i$. \\
        $T$ & Matrix ($|K|$ rows, $|I|$ columns), car insertion indicator in cell $k$ rank $i$. \\
        $E$ & Matrix ($|K|$ rows, $|K|$ columns), number of jockey moved between two cells. \\ \hline
        \end{tabular}
    \caption{Summary of the inputs, matrix of utility and matrix of car placements with two additional constants, expected by the greedy algorithm described in Algorithm~\ref{alg:ch3_greedy_algorithm} and its outputs, three matrices describing the actions the staff need to do.}
    \label{emphtab:ch3_summary_io_greedy}
\end{table}
    
The greedy approach described in Algorithm~\ref{alg:ch3_greedy_algorithm} is based on two lists of ``cars coordinates'' (a couple $(\mathit{cell}, \mathit{rank})$).
The first is called $\mathit{MC}$ (Movable Cars) to keep track of cars that could be moved represented by a list of couples $(\mathit{cell}, \mathit{rank})$.
The second one is called $\mathit{FP}$ (Free Places) to keep track of the free places where cars could be put represented by a list of couples $(\mathit{cell}, \mathit{rank})$ too.
Those two lists are first initialized and filled during the initialization part and then are used to set the values of matrices $F$, $T$, $E$ during the resolution part.

% TOKEEP: Check that to know how to split algorithm block in multiple parts: https://tex.stackexchange.com/questions/18949/algorithm2e-split-over-several-pages
\begin{algorithm}[!p]
    \footnotesize
    \LinesNumbered
    \DontPrintSemicolon
    % \setcounter{AlgoLine}{10}
    
    Set $\mathit{MC}$ to EmptyList~~\tcp{List of Movable Cars.}
    Set $\mathit{FP}$ to EmptyList~~\tcp{List of Free Positions.}
    Set $\mathit{nr}$ to 0.~~\tcp{Number of Relocation done.}
    
    \tcc{Initialize $\mathit{MC}$, $\mathit{PC}$ and $\mathit{FP}$ with relevant info from $\mathit{S}$.}
    \ForEach{cell $k$ in $K$}{
        Set $\mathit{rankCar}$ to highest rank from cars in $k$\;
        Set $\mathit{rankFree}$ to $\mathit{rankCar} + 1$\;
        \If{$0 \leq \mathit{rankCar}$}{
            Add ($k$, $\mathit{rankCar}$) to $\mathit{MC}$
        }
        \If{$\mathit{rankFree} < \nu$}{
            Add ($k$, $\mathit{rankFree}$) to $\mathit{FP}$
        }
    }
    
    \tcc{Try to move as much car as possible, by first moving cars with a rank too high first and then move the cars with lowest utility.}
    \While{$nr < \gamma$ and $\mathit{MC}$ not empty}{
        Set ($\mathit{carCell}$, $\mathit{carRank}$) to pair (cell, rank) of highest car rank from cars in $\mathit{MC}$\;
        \If{$\mathit{carRank} < \nu$}{
            \tcp{If there is no car to remove in priority, then move the one with lowest utility.}
            Set ($\mathit{carCell}$, $\mathit{carRank}$) to pair (cell, rank) of lowest car predicted utility from cars in $\mathit{MC}$\;
        }
        Set $\mathit{carUtility}$ to $U[\mathit{carCell}, \mathit{carRank}]$\;
        Set ($\mathit{destCell}$, $\mathit{destRank}$, $\mathit{destProfit}$) to tuple (cell, rank, profit) from the best profitable move possible of car ($\mathit{carCell}$, $\mathit{carRank}$, $\mathit{carUtility}$) to any location in $\mathit{FP}$ while taking into account relocation costs\;
        Remove ($\mathit{carCell}$, $\mathit{carRank}$) from $\mathit{MC}$\;
        \If{$\nu \leq \mathit{carRank}$ or $\mathit{destProfit} > 0$}{
            Remove ($\mathit{destCell}$, $\mathit{destRank}$) from $\mathit{FP}$\;
            Remove ($\mathit{carCell}$, $\mathit{carRank} + 1$) from $\mathit{FP}$\;
            \If{$\mathit{destRank} < \nu - 1$}{
                Add ($\mathit{destCell}$, $\mathit{destRank} + 1$) to $\mathit{FP}$\;
            }
            \If{$\mathit{carRank} > 0$}{
                Add ($\mathit{carCell}$, $\mathit{carRank} - 1$) to $\mathit{MC}$\;
            }
            Add 1 to $\mathit{nr}$\;
            Set $F[\mathit{carCell}, \mathit{carRank}]$ to 1\;
            Set $T[\mathit{destCell}, \mathit{destRank}]$ to 1\;
            Add 1 to $E[\mathit{destCell}, \mathit{carCell}]$\;
        }
        
    }
    Return F, T, E
    
    \caption{Greedy Algorithm}
    \label{alg:ch3_greedy_algorithm}
\end{algorithm}
        
The first part of the greedy algorithm initializes $\mathit{MC}$ by adding to it the ``coordinate'' of the car with the highest rank for each cell $k$ and in the same time initialize $\mathit{FP}$ by adding to it the ``coordinate'' of the lowest rank without a car for each cell $k$.
Thus $\mathit{MC}$ and $\mathit{FP}$ are initialized in order to help keep the constraints Equation~\eqref{eq:relocGap} and Equation~\eqref{eq:maxCarPerCell} valid during the whole process.
Then the second part of the greedy algorithm will try to move at most $\gamma$ cars.
First, the car of highest rank in $\mathit{MC}$ is taken from this list, if its rank is superior to the limit $\nu$ then it is a car to remove in priority from its cell to satisfy constraint Equation~\eqref{eq:maxCarPerCell}.
If not, then the car with the lowest expected utility is selected from $\mathit{MC}$.
Once a car to move has been chosen, the best destination is chosen, an empty space, in $\mathit{FP}$ to place this car, i.e find where it is possible to maximize the gain of expected utility while taking into account the cost induced by this relocation.
Since all the moves for this car are explored, it is removed from the list of movable cars $\mathit{MC}$.
If this car is to be moved in priority, i.e its rank is higher than the maximum $\nu$, or is profitable to relocate then a relocation will be done; otherwise nothing is done. 
In the case of a relocation, the free places list $\mathit{FP}$ is first updated by removing the ``coordinate'' of the destination where to place the car from it and by removing from $\mathit{FP}$ the empty space from the cell where the car comes from.
This is done to respect the constraint Equation~\eqref{eq:relocGap}.
Then if it would be still possible to place a car in the destination cell without going over the $\nu$ limit, the new lowest rank without a car is added to the list of free positions $\mathit{FP}$.
The last list update adds the car with the new highest rank from the source cell to $\mathit{MC}$ if there is still a car in the source cell once the previously selected car has been moved. 
Finally, the counter $\mathit{nr}$ is updated to add 1 to the number of relocation made and the output matrices to represent the actions done are updated.
Once $\gamma$ moves have been made or if there are no cars to move anymore, the matrices $F$, $T$ and $E$ are returned.


\subsection{Relocation Strategy Evaluation}
\label{sec:ch3_relocation_evaluation}

In this section four \emph{strategies} see their performance being evaluated.
First, the methodology proposed in Section~\ref{sec:ch3_jockey_relocation} is studied.
Then its performance is compared to two baselines: the historical revenue of the service and the greedy algorithm from Section~\ref{sec:ch3_greed_relocation}.
A variant of the proposed methodology is also studied to evaluate the impact of optimizing the fleet distribution before optimizing the jockey relocations.
The evaluation of those relocation methods is detailed, the comparison between the results returned by the relocation method and the historical revenue of the service is not trivial.
Those evaluations are made on the city services, \emph{Madrid}, \emph{Paris} and \emph{Washington}.

\subsubsection{Relocation Strategies}

In order to evaluate the \emph{Integer Linear Programming} approach to relocate the cars of the fleet to particular city cells in the morning, as described previously, the comparison between the expected revenue of four different strategies is made.
The first strategy is a ``naïve'' baseline relying on historical data, it is denoted by \emph{Historical}.
It is based on the historical fleet position every morning, and its true utilization during the day. 
The corresponding revenue is the number of driven minutes multiplied by the price paid by users for car renting.
The other baseline called \emph{Greedy} is more advanced and uses the greedy algorithm described in Section~\ref{sec:ch3_greed_relocation} to maximizes the car utilities while taking into account the costs to move the jockeys around the city.
Then the third strategy \emph{Optim DO} is the variant of the method described in Section~\ref{sec:ch3_jockey_relocation} and denotes a two steps (Double) Optimization strategy: the first one maximizes the fleet utility without taking into account relocation costs while the second one, after fixing the car location found, minimizes the jockeys' repositioning costs.
Finally the last strategy denoted by \emph{Optim SO} and described in Section \ref{sec:ch3_jockey_relocation} is evaluated.
It uses a joint (Single) Objective.
This objective maximizes the service utilization revenue minus the costs to reposition the jockeys. 

\subsubsection{Utility Estimation}

The ground truth utility used to evaluate the revenue earned by the service with the non-historical approaches is a mix between the historical utility and the predicted one.
When the \emph{Greedy} algorithm or the \emph{Integer Linear Programming} solver proposes a relocation solution different from the historical one, the predicted utility is used for the cars for which this value is unknown.
For example, if the \emph{Greedy} algorithm proposes to place 5 cars in cell $k$ and only 4 cars have been placed there historically, then the evaluation takes into account the historical utility for cars of rank 1 to 4 and the predicted utility for rank 5.

For all the strategies, it is assumed that the price $p$ paid by the user per driven minutes is constant.
Thus, the fleet expected revenue is the total utility multiplied by $p$.
The sweeper car cost $q_{kl}$, described in Section~\ref{sec:ch3_jockey_relocation}, is computed given the gross hourly salary of a jockey $c_j$ which has to be doubled (during a jockey repositioning from $k$ to $l$ there are two jockeys in the sweeper car: the one that needs to be repositioned and the driver of the sweeper car), the cost of running a sweeper car per km $c_s$, the average speed (in km/h) of a car inside the city $s$ and the distance $d_{kl}$ (in km) between the cell $k$ where a jockey is picked up by the sweeper car and the cell $l$ where he/she is dropped off :

    $$
    q_{kl} = \frac{d_{kl} \cdot (2 \cdot c_j)}{s} + d_{kl} \cdot c_s 
    $$

Furthermore, an additional hypothesis is made that there are at most 70 relocations each night, so $\gamma = 70$, which corresponds to 7 jockeys that can do 10 relocations on average per night.

\subsubsection{Case Studies}

The evaluation of the proposed \emph{ILP} formulation is done for the three services that are presented in Chapter~\ref{ch:background}.
The objective is to evaluate the revenue that the service can expect by applying the proposed relocations to its fleet.
This is done for the \emph{Greedy} baseline, the proposed \emph{ILP} formulation \emph{Optim SO} and its variant \emph{Optim DO}.

From now on \emph{solver car assignment} denotes the optimal solution to the optimization problem found by the proprietary solver Gurobi. 
The utility (number of driven minutes) is estimated using the best regression model found in the previous experiment.
Thus in the case of \emph{Madrid}, a \emph{K-Medoids} clustering of the city cells is followed by a \emph{GBR} regressor, using the feature set \emph{All}.
In the case of \emph{Paris}, a \emph{K-Means} clustering is applied followed by a \emph{GBR} regressor, using the feature set \emph{All}.
Finally, for \emph{Washington}, we chose a \emph{K-Means} approach with a \emph{GBR} regression model using the feature set \emph{All}.

For all approaches, the total daily revenue, costs included, expected by the three services are computed.
These are plotted in Figure~\ref{fig:ch3_optim_madrid} for \emph{Madrid}, in Figure~\ref{fig:ch3_optim_paris} for \emph{Paris} and in Figure~\ref{fig:ch3_optim_washington} for \emph{Washington}.
In those figures, one plot is associated with one week of the test set, one point being associated with the revenue of a particular relocation strategy for one day.
For each day, the \emph{Historical} baseline has a base value of 100 to help make a relative comparison to the other strategies.
Besides, the averages per day are reported for the three datasets in Table \ref{tab:ch3_optim_table}.
It is expected that \emph{Optim SO} should give a larger revenue than \emph{Optim DO} since the one-step strategy can take into account the costs while relocating the car which should help reduce the number of unnecessary relocations costs.
Furthermore, it is expected that the \emph{Greedy} approach should yield a lower revenue than \emph{Optim SO} because of the sub-optimal solution found by the greedy algorithm versus an optimal solution found by a \emph{Integer Linear Programming} solver.

It should be noted that for the three services, all strategies converged to a solution in less than 5 minutes each, making the three strategies usable by an operational team for a real service.
Furthermore with the \emph{ILP} formulation given in Section~\ref{sec:ch3_fleet_relocation} in the case of strategies \emph{Optim SO} and \emph{Optim DO}, the number of decision variables used is estimated to be $|K|^2 + 2 \cdot (|K| \cdot |I|)$ and the number of constraints is estimated to be $3 \cdot (|K| \cdot |I|) + 3 \cdot |K| + 3$.
Thus in the case of \emph{Madrid}, for a day in the worst-case scenario with $|K| = 155$ cells and a maximum car rank $|I| = 30$ encountered in the data, the solver has to find the values of $33\,325$ decisions variables with the help of $14\,418$ constraints.
For \emph{Paris} with $209$ cells and a maximum observed rank of $51$, in the worst-case scenario the solver had to find the values of $64\,999$ decision variables with the help of $32\,607$ constraints.
Finally, for \emph{Washington} with $411$ cells and a maximum observed rank of $26$, the worst-case scenario implies the use of $190\,293$ decision variables and $33\,294$ constraints.

\begin{table}[!b]
    \centering
    \small
        \begin{tabular}{|l|l|l|l|}
        \hline
        \multirow{5}{*}{Madrid} & Strategy & Revenue & Nb Relocation \\ \cline{2-4} 
            & Historical & 100 & N/A \\ \cline{2-4} 
            & Greedy & 106 & 70 \\ \cline{2-4} 
            & Optim DO & 105.9 & 70 \\ \cline{2-4} 
            & Optim SO & \textbf{106.8} & 70 \\ \hline
        \hline
        \multirow{5}{*}{Paris} & Strategy & Revenue & Nb Relocation \\ \cline{2-4} 
            & Historical & 100 & N/A \\ \cline{2-4} 
            & Greedy & 101.8 & 62 \\ \cline{2-4} 
            & Optim DO & 88.4 & 70 \\ \cline{2-4} 
            & Optim SO & \textbf{105.1} & 62 \\ \hline
        \hline
        \multirow{5}{*}{Washington} & Strategy & Revenue & Nb Relocation \\ \cline{2-4} 
            & Historical & 100 & N/A \\ \cline{2-4} 
            & Greedy & 99.6 & 70 \\ \cline{2-4} 
            & Optim DO & 95.1 & 70 \\ \cline{2-4} 
            & Optim SO & \textbf{102.9} & 70 \\ \hline
        \end{tabular}
    \caption{Daily average revenue and daily average number of relocations for four relocation strategy for the three services.}
    \label{tab:ch3_optim_table}
\end{table}

\paragraph{Madrid.} 

Table~\ref{tab:ch3_optim_table} shows that for this service, on average for each day all the optimization strategies increase the revenue of the company. 
On average for each day, approaches \emph{Greedy} and \emph{Optim DO} show a potential increase of 6\% and 5.9\% respectively while the proposed approach \emph{Optim SO} makes a potential increase of 6.8\%.
Even if the difference between \emph{Greedy} and \emph{Optim SO} seems low, it should be noted that as stated earlier \emph{Madrid} is heavily used and a sole increase of 0.8\% makes a noticeable difference for the service operator.
Furthermore, it explains better why the three approaches have comparable performances, since the service is heavily used the costs induced by employing jockeys to relocate the cars are negligible compared to the possible revenue increase.
Besides, the three strategies use the jockeys to the maximum, meaning that it may even be possible to relocate other cars and be still profitable by employing more jockeys.

\begin{figure}[!t]
    \centering
    \includegraphics[width=1\textwidth]{figure/ch3_optim_madrid.jpeg}
    \caption{Daily revenue expected by \emph{Madrid} (one x-axis tick is a day, one plot per week of the test set), with the \emph{Historical} strategy (dashed blue line) as the base index. The red full line, the dashed yellow and dotted mauve lines are respectively the normalized revenues provided with \emph{Optim SO}, \emph{Optim DO} and \emph{Greedy} strategies.}
    \label{fig:ch3_optim_madrid}
\end{figure}

For further details, Figure~\ref{fig:ch3_optim_madrid} shows four plots for the four weeks in the test set.
As stated in Table~\ref{tab:ch3_optim_table}, the three approaches have almost similar performances.
It can be noted that the approach \emph{Optim SO} manages to stay above an improvement of 5\% for more than 75\% of the days in the test set.
Besides, that approach is under the \emph{Historical} baseline only for one day (2018-09-22) over the four weeks in the test set.

\paragraph{Paris.}

The results for this service are shown in Table~\ref{tab:ch3_optim_table}.
The average daily revenue should both be increased by the use of the \emph{Greedy} and \emph{Optim SO} approaches, by respectively 1.8\% and 5.1\%, while the approach \emph{Optim DO} shows a potential daily average \emph{decrease} of 11.6\%.
These results are coherent with the average daily number of relocation made : both \emph{Greedy} and \emph{Optim SO} are capable of choosing the right number of vehicles to relocate when \emph{Optim DO} is not because of its conception.
Indeed, it seeks to relocate as many cars as possible without studying if each relocation is ``profitable'', i.e. the relocation costs less than the expected usage for the next day.

\begin{figure}[!t]
    \centering
    \includegraphics[width=1\textwidth]{figure/ch3_optim_paris.jpeg}
    \caption{Daily revenue expected by \emph{Paris} (one x-axis tick is a day, one plot per week of the test set), with the \emph{Historical} strategy (dashed blue line) as the base index. The red full line, the dashed yellow and dotted mauve lines are respectively the normalized revenues provided with \emph{Optim SO}, \emph{Optim DO} and \emph{Greedy} strategies.}
    \label{fig:ch3_optim_paris}
\end{figure}

Figure~\ref{fig:ch3_optim_paris} shows how the three approaches perform each day for the whole 5 weeks of the test set.
One can notice that while the \emph{Greedy} strategy performance is consistent in being slightly better than the \emph{Historical} baseline, it is not the case of \emph{Optim DO} which is heavily influenced by the car usage. 
Indeed, since \emph{Optim DO} seeks to relocate as many cars as possible before minimizing jockey costs, it is often relocating cars that are not profitable to move once the jockey repositioning is taken into account.
Meanwhile, the approach \emph{Optim SO} is most of the time over the \emph{Historical} baseline, only three days have a lower revenue, and always above the \emph{Greedy} baseline.
However, those results should be mitigated by the low performance of the regression models in the case of \emph{Paris}, with more accurate prediction of the car utility the three relocation strategies would be more accurately evaluated.

\begin{figure}[!t]
    \centering
    \includegraphics[width=1\textwidth]{figure/ch3_optim_washington.jpeg}
    \caption{Daily revenue expected by \emph{Washington} (one x-axis tick is a day, one plot per week of the test set), with the \emph{Historical} strategy (dashed blue line) as the base index. The red full line, the dashed yellow and dotted mauve lines are respectively the normalized revenues provided with \emph{Optim SO}, \emph{Optim DO} and \emph{Greedy} strategies.}
    \label{fig:ch3_optim_washington}
\end{figure}

\paragraph{Washington.}

The last part of Table~\ref{tab:ch3_optim_table} shows the average daily revenue and number of relocation of the three approaches evaluated.
Unlike the two previous services, \emph{Washington} is spread on a bigger area with its $411$ cells.
With a utilization comparable to \emph{Paris} but a bigger surface to cover, the costs associated with the use of jockeys to relocate to cars are expected to be higher.
This is verified by the average results in the table, with the \emph{Greedy} and \emph{Optim DO} approach, it seems not be possible to beat the historical baseline.
Since the strategy \emph{Greedy} does not optimize the costs induced by repositioning the jockeys, its average performance stays around the baseline.
As for \emph{Optim DO}, since the cars are first selected before any cost is taken into account, this means that some cars that have a very low predicted utility are moved to a very high utility empty spot and seems profitable.
But when taking into account the costs, this apparent profitability is countered by the costs.
However our proposed strategy \emph{Optim SO} should give a daily average increase of 2.9\% of the revenue during the 4 weeks of the test set.
While the three approaches relocate as many cars as permitted, \emph{Optim SO} is capable of selecting the best cars to relocate compared to \emph{Optim DO} and to better optimize the jockey repositioning compared to \emph{Greedy}.

Figure~\ref{fig:ch3_optim_washington} gives a precise view of the three strategies performance.
While \emph{Optim DO} seems capable of beating the historical baseline only once (2020-01-09), our approach \emph{Optim SO} seems to yield a lower revenue than the baseline seven days over the fours weeks of the test set.
As in the case of \emph{Paris}, those results should be mitigated by the poor prediction performance of the regression models.

\FloatBarrier
\section{Conclusion}
In this chapter, a methodology has been proposed to optimize both the placement of the fleet during the night and the trips jockeys need to take to go from a relocated car to another one.
This methodology is based on two steps, first it predicts the future utilization of cars in every part of the city and second its make the optimization.
Two experiments have been made to assess the performance of the methodology.

The first experiments have evaluated the error rate of regression models for the three case studies.
In each case, the utilization of a \emph{Gradient Boosting Tree Regressor} led to a lower error rate than the use of a mean value of the utility prediction.
For both \emph{Paris} and \emph{Washington}, this regression model has a high error rate ($\geq$ 70\%) when compared to the historical utility.
It is explained by both the high variability of the measure of the daily utility: it depends on the chain of trips during the day for each car.
Over a single day, the randomness associated with the car usage leads to different chains of the trip even when the car is always placed in the same location.

The second experiments have evaluated the daily utility expected with the placement optimized by the proposed methodology.
In all cases, the best regression model has been used to predict the expected utility from placing cars in every location of the city.
The experiments shown that for the three case studies, the proposed methodology increased the daily expected profit when compared to the historical one.
The study of the long-term effects of using the methodology has yet to be evaluated and is the subject of the next chapter.


% The first approach both evaluated the performance of the daily utility prediction by the regression models and evaluated the gains that could be expected by the \emph{Integer Linear Programming} (\emph{ILP}) optimization model.
% The evaluation shown that with all the available feature, for \emph{Madrid} in the best case the utility prediction error represented 33\% of the daily usage in each cell and for \emph{Paris} and \emph{Washington} this error rate was respectively 74\% and 87\%.
% When compared to the baseline that predicts for each car the mean utility depending on the car rank, in the bast case the utilization of regression models made an improvements of 3\% for \emph{Madrid}, 12\% for \emph{Paris} and 34\% for \emph{Washington}.
% For the three case studies, the utilization of such regression models to predict the daily utility of cars is justified.
% Then experiments evaluated the solution proposed by the \emph{ILP} model against the historical utility performance of the service, such that the utility made by the historical placement was compared to the one that was expected from the placement found by the \emph{ILP} solver.
% The results shown an improvement of 6.8\% in total daily utility for \emph{Madrid}, 5.1\% for \emph{Paris} and 2.9\% in the case of \emph{Washington}.
