On the Simulation of LoRaWAN Networks: A Focus on Reproducible Parameter Configuration

: In the past decade, with the rapid proliferation of Internet of Things (IoT) devices new applications have emerged in diverse fields such as agriculture, smart cities, and healthcare. In this context, LoRaWAN, a Low-Power Wide Area Network protocol, has gained significant adoption. Its advantages, including low-power consumption and long-range communication capabilities, make it an ideal solution for IoT communication. Simulation tools have played a crucial role in facilitating experimentation and advancing the implementation of the LoRaWAN protocol. However, effectively modelling these experiments present a significant challenge. Through a comprehensive survey of the literature on LoRaWAN, we have identified variations in parameter configuration among different works and observed a lack of sufficient information in most of them, thereby impeding result reproducibility. This paper offers a comprehensive overview of LoRaWAN technology, encompassing its architecture and underlying technologies such as LoRa and LPWAN. Furthermore, we present and compare the main simulation tools currently available for LoRaWAN, discuss the parameters that significantly impact the performance of a LoRaWAN network, and explore how researchers have configured these parameters to model their simulation experiments.


Introduction
During the last decade, the Internet has gone through a revolution in terms of network organization and paradigms.Nowadays, we are able to design and visualize everyday devices as connected things.The paradigm of the Internet of Things (IoT) has leveraged new applications and the need to rework the traditional network layers.This promising scenario has attracted the attention of both academy and industry [1].
This new era of millions of internet-connected battery-powered devices presents several challenges, including communication range, scalability, network flexibility, and power management.These challenges have given rise to innovative solutions at both the physical and link layers [2].One promising solution for meeting these challenges is Low-Power Wide Area Network (LPWAN), which aligns well with the requirements of IoT applications.LPWAN protocols provide low data throughput, ensuring low power consumption [3].Additionally, LPWAN technologies are capable of transmitting data between devices at distances of up to 10 km [4].
There are a wide range of applications based on LPWAN, varying from healthcare and agriculture [5][6][7] to disaster planning using geographical and meteorological sensing [8].Different communication protocols were developed to meet the requirements of LPWAN, such as Sigfox [9], LoRaWAN [10] and LTE-M [11].The next few years will see even greater developments in wireless networks, with the roll-out of 5G networks [12] and the emergence of 6G [13].This major revolution in wireless communication will bring new technologies at all levels, from the physical layer to real-time applications.As a result, communication technologies, including LoRaWAN, will be impacted with new paradigms such as network slicing, cloud Radio Access Network (cloud-RAN), and heterogeneous networks becoming part of the future landscape [14].
The use of simulation tools has proven to be a low-cost solution to collect data in different scenarios, providing studies of potential projects and being essential for the development of wireless networks.In addition, during the planning of a project, simulation tools allow us to ensure that the correct technology has been chosen [15].This paper aims to provide a comprehensive overview of LoRaWAN architecture, the main simulation tools for this technology, and a comparison of how crucial parameters are configured in both simulation and real-world experiments found in the literature.By addressing the lack of detailed methodology explanations in much of the experimental work in LoRaWAN, this paper will assist researchers in selecting appropriate simulation tools and configuring simulation parameters effectively.We are confident that the insights presented in this paper will greatly benefit researchers in this field.

Related Work
Although LoRaWAN is a relatively new technology, it has gained significant popularity along with its surrounding technologies (LoRA, LPWAN) in recent years.This is due to factors such as ease of deployment, scalability, accessibility, and rapid industrial adoption [16].The academic community has responded by producing numerous works in the area.To gain a comprehensive understanding of how this technology is explored in the literature and to compare it to related topics, recent surveys were selected and analyzed.Particular attention was paid to the simulation aspects of LoRaWAN technology.To provide a comprehensive overview of recent surveys in the field, we have classified them into three categories: (i) those covering various LPWAN technologies, (ii) those specifically exploring the LoRa PHY layer, and (iii) those concentrating on LoRaWAN.
LPWAN: Studies on LPWAN aim to characterize the different technologies that use this type of network, such as Sigfox, LoRaWAN and NB-IoT.For example, Ayoub et al. [17] explore the various aspects of LPWAN network technologies and their suitability for applications with mobility requirements.Another study by Jonathan et al. [18] compares LoRaWAN with other proprietary LPWAN network technologies.Aldahdouh et al. [19] analyses LPWAN as a 5G candidate, in comparison with other emerging technologies (NB-IoT, Sigfox, and LTE-M).Triantafyllou et al [20] provides a more general overview by comparing LPWAN (including LoRaWAN) with other wireless communication technologies.
LoRa: Studies on LoRaWAN often focus on its proprietary physical layer mechanism, LoRa.Lavric et al. [21] provide an overview of both LoRa and LoRaWAN technologies, with a focus on LoRa modulation and the main challenges of the technology.Saari et al. [22] conducted a survey to categorize the current state-of-the-art of LoRa-based technologies.Recently, Sarker et al. [23] took a new approach to LoRa by analyzing the literature and proposing an architecture to integrate edge computing features into IoT-based devices.
LoRaWAN: The literature on LoRaWAN technology has seen a number of significant contributions.Haxhibeqiri et al. [24] conducted a comprehensive analysis of the technology, exploring its network layer and proposing possible protocol enhancements.Khutsoane et al. [25] reviewed key parameters affecting the feasibility of LoRaWAN in wireless fields.Adelantado et al. [26] analyzed the limitations of the technology.Several recent studies have focused on exploring use cases and application scenarios for LoRaWAN [26][27][28].Additionally, the technology has been evaluated in terms of energy efficiency [29] and feasibility in a massive IoT world [30].LoRaWAN is also studied as a strong candidate and enabler for several applications, such as Tracking [17,22,[25][26][27][28], Health Care [23,24], Agriculture [30], Smart Vehicles [31,32] and Smart Cities [33,34].In the area of LoRaWAN simulations, Almuhaya et al. [35] and Bouras et al. [36] provided an overview of the technology, analyzing current simulation tools based on aspects such as operating system, language, community support, installation requirements, and number of previous work that used such tools.Another study [37] focused in one specific simulator (ns-3) and explored the different LoRaWAN simulation modules available for it, considering features and limitations.
Unlike prior work [35][36][37], this paper offers a thorough analysis of LoRaWAN, simulation tools, and propagation models and focuses on examining the ways in which simulations are conducted by researchers based on important parameters.

Contributions
This survey offers a novel perspective on the literature concerning LoRaWAN technology and simulation experiments.Its main contributions include:  providing an overview of LoRaWAN, with a focus on its architectural components, and introducing a classification of Network Servers into four distinct categories;  reviewing and comparing the primary simulation tools available for LoRa and LoRaWAN;  reviewing the main channel propagation models used for LoRa simulation;  reviewing the key parameters that have a significant impact on the performance of LoRaWAN networks and comparing how authors of both testbed and simulation-based studies configure their values.

Paper Organization
Figure 1 presents the structure of this paper.The remaining sections are organized as follows: Section 2 provides an overview of LPWAN networks; Section 3 presents a comprehensive overview of the LoRaWAN network architecture; Section 4 review and compares the main simulation tools for LoRa and LoRAWAN; Section 5 presents the channel propagation models commonly used for LoRaWAN simulation; Section 6 explores the key parameters that significantly impact the performance of LoRaWAN networks and their configuration in experimental scenarios; finally, Section 7, draws conclusions and outlines future research directions.

LPWAN Overview
Low-Power Wide Area Network (LPWAN) is a type of WAN that enables long-range communication between devices while conserving power by transmitting at a low bit rate [38].Unlike a wireless WAN, which is designed to connect users across a geographic region, LPWAN focuses on connecting IoT devices.Figure 2 provides a brief comparison between LPWAN and other communication technologies.[39] LPWAN has garnered significant attention in recent years due to its rapid adoption in IoT deployment for businesses around the world.This trend is expected to continue, as IoT market reports predict that the number of connected IoT devices will increase from 7 billion in 2018 to over 22 billion in 2025, with LPWAN devices growing at a rate of more than 100% annually [40].

Figure 2. LPWAN and other network communication technologies
The communication range in LPWAN is typically 5 to 40 km, depending on environmental conditions, which results in a reduced number of access points when compared to other technologies.The low power consumption of LPWAN also enables battery-powered devices to operate for more than 10 years.Additionally, end nodes are usually much less expensive due to their simple design and low-cost antennas.
LPWAN technology has become an important enabler in the growth and evolution of the IoT market, offering solutions for long-range, low-power communication between devices.The different LPWAN solutions available in the market, such as SigFox, LoRAWAN and NB-IoT, have their own technical features and tradeoffs, making them suitable for specific IoT applications.Further analysis and comparisons, such as the one conducted by Mekki et al. [41], help in understanding the distinctions among these solutions.In this paper, our focus is on LoRaWAN technology.

LoRAWAN Overview
LoRaWAN is a communication protocol built on top of the LoRa Physical Layer and is a leading LPWAN technology in the IoT industry.One of the reasons for its broad adoption is that it uses an unlicensed range of radio frequency spectrum [22], which avoids the spectrum congestion of other well-known frequency ranges, such as the one used by cellular network companies.LoRaWAN addresses the challenge of region restrictions by being specified for a number of bands for each region, which directly impacts the backend infrastructure.
The modulation and physical layer protocol used result in a long-range communication with low data rate, enabling low energy consumption for battery-operated IoT devices [18].The LoRa Alliance [42], a non-profit association of companies, standardizes the MAC features of LoRa, promoting the development and adoption of this technology by means of an open standard for LPWAN solutions.
Table 1 presents a comparison of LoRAWAN and other LPWAN technologies.As we can see, battery lifetime, power efficiency, and interference immunity are some of the aspects in which LoRaWAN stands out.This, in addition to the long range coverage and the community involvement, are the main reason behind the huge adoption of this technology for IoT solutions [42].The following sections provide more information on the technical aspects of LoRaWAN.

LoRA
LoRaWAN is built on top of the proprietary physical layer technology known as LoRa (Long-Range) [42].This technology utilizes Chirp Spread Spectrum (CSS) modulation for data encoding.In the field of communication technology, there are several ways to encode data at the physical level using different modulation mechanisms.These mechanisms differ in the way the carrier signal is manipulated to represent digital data.Some of the most well-known modulation techniques include:  Frequency Shift Keying (FSK): the frequency of the carrier signal represents the digital data being transmitted;  Phase Shift Keying (PSK): the phase of the carrier signal is modified to represent either 0 or 1;  Chirp Spread Spectrum (CSS): is a specific spread spectrum mechanism where a signal is spread in the frequency domain.A chirp is a signal that increases (up-chirp) or decreases (down-chirp) its frequency over time, and the original message is encoded in these chirps.
As previously noted, LoRa is based on CSS and is a proprietary modulation scheme patented by SemTech [43].The unique feature of this modulation technique is that it provides long-range communication with a low energy consumption, making it ideal for battery-powered devices used in IoT networks [21].In urban environments, the range of LoRa signals can vary from 10 to 15 Kms.However, the maximum distance that a LoRaWAN packet has traveled was 766 Kms during an experiment where a packet sent from Ariza, Spain was received by a gateway in Grenoble, France. Figure 3 illustrates an example of a LoRa signal using different Spreading Factors (SF).As can be seen, the signal continuously spans the channel and changes its frequency over time.The SF determines the length of the chirp [44], with an SF of 7 representing the shortest transmission time and an SF of 12 representing the longest transmission time for the signal.Generally, a lower spreading factor results in a shorter transmission time but improved resistance to interference, while a higher spreading factor results in a longer transmission time but reduced resistance to interference.Despite being a proprietary technology, there have been efforts in the research community to decode the LoRa signal and gain a deeper understanding of the modulation scheme.For instance, Knight et al. [45] have explored the modulation and encoding components of LoRa and have presented an open-source version of LoRa based on their findings.

Communication Architecture
This section describes the network architecture of LoRaWAN.LoRaWAN networks employ a star-of-stars topology.In essence, end devices transmit data to gateways, which then forward it to the network server.The collected data can then be accessed by an application server, making it available to the end application.Conversely, the network server can also send data to end nodes, which will be forwarded to them through the gateways [42].An overview of the architecture is presented in Figure 4.
In order to comprehend the intricacy of each component in the LoRaWAN architecture, a bottom-up approach will be utilized in the following susbsections, beginning with the end node devices and progressing towards the application server.

End Devices
End devices, also referred to as nodes, are responsible for collecting and transmitting the useful data to the gateways via LoRa.These nodes are not tied to a specific gateway, so when data is transmitted, it is received by every gateway within the communication range of the end device, and then forwarded to the network server [42].
There are different implementations of LoRaWAN devices, with varying differences in how the link channel is accessed, which has a direct impact on energy consumption.The LoRa Alliance defines three classes of devices: Class A, Class B, and Class C.
Class A is a mandatory class for every end device.It supports bidirectional communication, allowing for both sending messages to the gateway and receiving messages from it.Class A devices send messages at any time.After the transmission, the device opens two receive windows at 1s and 2s to listen for server replies.
Class B extends the capabilities of Class A devices by adding beacon synchronization with the network server, enabling the scheduling of additional receive windows to receive downlink packets.
Class C devices have an always-open receive window, unless they are transmitting a message.This makes communication from the network server to the end devices more efficient, but also results in increased energy consumption due to the always-open receive window.

Gateway
Gateways serve as intermediaries in the transfer of data from end devices to the Network Server.This process involves the collection of messages sent by LoRaWAN-compliant devices and their subsequent transfer to high-bandwidth networks such as 4G, Wi-Fi, or Ethernet.
An analogy can be drawn between gateways and routers in a Wi-Fi network.However, gateways are equipped with a LoRaWAN concentrator, which can take the form of a board that can be connected to embedded devices such as Raspberry Pi [46] or BeagleBone Black [47].There are also LoRaWAN gateways available on the market, such as those produced by MikroTik (https://mikrotik.com/product/wap_lr8_kit##fndtn-specifications) and Cisco (https://www.cisco.com/c/en/us/products/routers/wireless-gateway-lorawan/index.html).
The level of control required determines the complexity of the gateway, and thus they can be classified into two categories: Minimal Firmware: are usually low cost and easy to set up; they are simple devices which are only capable of forwarding the packets; Operating System: are typically more sophisticated devices that run a complete operating system, with the packet forwarder being one of the tasks it performs.This allows the network administrator to choose the software used for packet forwarding, making packet management simpler and more flexible.

Network Server
LoRaWAN devices operate differently from the ones based on the IP protocol stack, utilizing an alternate protocol to transmit data to gateways.As a result, a separate component must handle the processing and IP protocol-based routing of that data.
Network Servers form the backbone of a LoRaWAN network, taking on the role of integrating all network com-ponents, from devices to applications [42].Network Servers vary in implementation and cater to various industry requirements.After conducting a review, we have classified Network Servers into four categories: Open-Source Network Server, Gateway Embedded Network Solution, Community Network Server, and Private Network Server.
Open-Source Network Server: provides full control over its source code and often relies on contributions from its user community.The benefit of this type of solution is the ability to customize, certify, and manage the entire network to meet specific application requirements.However, it comes with a high level of complexity, including the need for technical expertise for installation and configuration.Additionally, there may be a lack of specialized support, and some LoRaWAN features may not be fully developed.ChirpStack [48] is an example of an Open-Source Network Server.
Gateway Embedded Network Solution: is a type of Network Server that is integrated into the gateway itself.With this solution, the user typically purchases a gateway and can easily set up and deploy the Network Server, allowing for the management of devices connected to the gateway and data transmission to the application server.UrsaLink [49] is an example of this type of Network Server.However, in complex real-world deployments, this solution may have limitations such as scalability issues, limited LoRaWAN features (such as geoservers), the need for physical access to the gateways, a lack of monitoring mechanisms, and limited control over the network [50].
Community Network Server: is a decentralized solution where Network Servers are located globally and are available for free use by any user.This allows users to manage their devices and gateways using the infrastructure provided by the Network Server.This type of solution is usually offered by IoT companies, but may come with limitations such as limitations on the number of gateways/devices, on the access to logging and monitoring features, quality of service (QoS), and others.However, it provides users with a fast and straightforward way to access the LoRaWAN network, making it a suitable option for academic research and proof-of-concept projects.Loriot [51] is a well-known example of a Network Server solution that provides a Community Server option for its members.
Private Network Server: provides increased control and security, tailored to the specific requirements of the client.Its primary purpose is to offer full control over the network, customized to meet the specific needs of the client, as determined by their subscription.

Application Server
The data generated by end devices is transmitted through the network to reach the application server, which serves as the final destination.This server provides a platform for managing devices, applications, and organizations, and can also be accessed by other end applications using various methods.Typically, an application server is integrated into a network server solution and offers multiple mechanisms for clients to retrieve the data from sensors.Below, we give a brief overview of two popular connectivity mechanisms that are used, Message Queuing Telemetry Transport (MQTT) and HyperText Transfer Protocol (HTTP) [52].
MQTT protocol is an ISO standard for communication based on a publish-subscribe model [53].It works as an alternative to traditional client-server communication, in a sense that in MQTT, publishers and subscribers never communicate directly to each other.The communication is interfaced by the broker, a third element introduced by MQTT that is responsible for managing and filtering the messages going through it.This decoupling logic gives the protocol flexibility in terms of time, space and synchronization.MQTT is optimised for remote communications on low bit rate networks, making it one of the most popular communication protocol for IoT solutions [54].In MQTT, both publishers and subscribers are clients, while the broker assumes the server rule.A client is responsible for a series of operations, such as opening/closing network connection with the server, subscribing/unsubscribing to receive application messages from the server.The messages transmitted contain payload data, QoS, collection of properties and a topic name.The server (broker) is responsible for managing the communication between the MQTT clients (subscribers and publishers), which includes receiving, filtering, and sending the data.Authentication and authorization is also part of the broker's responsibility.In addition, the MQTT broker can be embedded into the gateway or in the cloud, increasing the flexibility of communication.From LoRAWAN perspective, MQTT is a fine selection to tie the communication between all the clients, which could be represented by LoRAWAN Gateways and applications.The sensors can forward their data to the gateways using LoRA, and those gateways will translate the messages into MQTT messages and forward them to the broker, hosted on the application server, which will manage it and send to the corresponding clients.

NS-3
NS-3 is a popular simulation tool, specially valuable to educational and academic purposes.It is a discreteevent network simulator that is implemented in both C++ and Python and it is backed by a robust community support and documentation.Furthermore, it encompasses a wide range of protocols, both Internet-based and otherwise [56].NS-3 is created as a collection of libraries or "models" that simulate various aspects of a network.These include channel propagation, mobility of end nodes and gateways, PHY and MAC layers, and more.The simulation results can be viewed through animations and external tools, as NS-3 has a command-line based user interface [56].
The modular architecture of NS-3 is another key advantage that allows users to enhance the software by fixing bugs or adding new features.Scientific contribution plays a crucial role in technological advancement and has great significance in the networking industry.The more collaboration among researchers, the quicker new tools and updates emerge.NS-3 serves as a prime example, with its frequent release of updated models and protocols each year, reflecting the active involvement of its user community [37].
In this section, we will examine some of the LoraWAN networking modules designed for the NS-3 simulator as described in [37].We will cover the main aspects of each module.It is noteworthy that all four modules only simulate class A devices.

Module I
The purpose of this module (https://github.com/networkedsystems/lora-ns3) is to promote studies and advancements in LoRa technology with class A devices through NS-3 simulation.It implements features such as support for distributed gateways connected via an IP network to a controlling server, classes for easy application installation and new MAC layer commands, and a class to monitor the energy consumption of LoRa protocol's physical layer states (LoraRadioEnergyModel) [57].
The module's architecture, implemented classes, and public code repository are thoroughly explained in its publication, along with demonstrations of three simulation scenarios with varying parameters.The authors emphasize the use of the module to address issues such as the impact of different spreading factors, packet interference, network reliability and scalability, among others.

Module II
Van den Abeele et al. [58] propose combining a LoRa error model with the LoRaWAN protocol's MAC layer to create a module for NS-3.The primary objective of their study is to examine the scalability and capacity of LoraWAN networks as the number of gateways and devices grows.The error model serves as an interference model, and its formulation involves intricate simulations of bandwidth and bit error rates.
The proposed module (https://github.com/imec-idlab/ns-3-dev-git/tree/lorawan)facilitates the simulation of scenarios involving multiple gateways and devices, different types and patterns of traffic, variations in data rates, retransmissions, and reception parameters.The paper [58] shows that increasing the density of gateways can reduce the negative impact of downstream traffic on the delivery rate of confirmed upstream traffic, but it cannot completely eliminate it.The authors also provide a comprehensive description of the module, access to its repository, and some example code, although some of the code is not functional.

Module III
The module (https://github.com/drakkar-lig/lora-ns3-module)proposed by To and Duda [59] was created as part of their research aimed at enhancing the performance of LoRa devices while maintaining their low power consumption and improving the accuracy of LoRaWAN network simulations in NS-3.To validate the module, the authors compared the results obtained from the module with metrics from a real-world testing environment.
The enhancements made to the module to minimize power loss include the implementation of the Carrier Sense Multiple Access (CSMA) and CSMA-x protocols.CSMA involves checking if the channel is being used by another transmission before attempting to send a packet, a technique commonly referred to as Listen Before Talk (LBT).The CSMA-x protocol extends the CSMA approach by requiring the device to test the channel for a specified interval, x, and if there is no activity, it can attempt a transmission.
The authors claim that the module accurately simulates the capture effect, which reduces the rate of packet loss due to collisions.The simulation results obtained with the implementation of CSMA show a reduction in the collision rate, although there is a moderate increase in the power consumption of the devices [59].However, the article lacks actual description of the CSMA and CSMA-x protocols implementation, and the module code only allows for a single gateway, with limited documentation.

Module IV
The module (https://github.com/signetlabdei/lorawan)by Magrin et al. [60] was developed with the aim of evaluating the performance of a LoRaWAN network in an urban environment.It is based on the consolidation of two key models: the LoRa physical layer, which represents the behavior and transmission of chirps, and the MAC layer.
This module includes the following features: simulations with multiple gateways and devices that have infrequent access to a wireless channel, the Adaptive Data Rate (ADR) algorithm, a basic network server, and functions to calculate statistics after the simulation.It is important to note that the module does not support network configurations outside of Europe, and there are no models to evaluate interference between partially overlapping channels or to implement certain MAC layer commands.
In their initial study [60], the authors highlight a significant improvement in coverage and uplink reliability when simulating the LoRaWAN network with a larger number of gateways.The results also showed that LoRaWAN has higher throughput compared to ALOHA due to the partial orthogonality of its spreading factors.
The authors have also published additional related work [61,62], which present solutions to additional problems that have been integrated into the source code.This module is constantly being improved, with a large user base and extensive documentation, including code examples and a NS-3 user mailing list.As a result, it is currently the most advanced module of its kind.

Flora
Flora (https://flora.aalto.fi/)was created as part of a research project aimed at exploring adaptive mechanisms in LoRA parameter configuration.It is a simulation framework written in C++ and based on the popular network simulation tool OMNeT++.Flora offers a range of features, including the design of LoRa networks (end nodes, gateways, and network servers), bidirectional communication, PHY and MAC layers, as well as the ability to simulate a backhaul network and evaluate its energy consumption [63].
The user interface of Flora is inherited from OMNeT++ and allows the network to be visualized.Additionally, example scenarios are included as part of the simulation tool.

LoraSim
LoraSim (https://github.com/adwaitnd/lorasim) is among the earliest simulations tools for LoRa technology [64].It uses SimPy, a discrete event simulation framework built on the Python programming language.In this simulator, end nodes can be either randomly positioned or arranged in a two-dimensional grid.The tool offers four distinct simulation types, each featuring different network properties, such as node and antenna configurations.The number of end nodes, LoRa networks, gateways, and simulation duration are among the parameters that can be altered.Additionally, the simulation produces several key outputs, including the Data Extraction Rate (DER) and Network Energy Consumption (NEC).

Phy Simulator
The Spreading Factor determines the number of chirps that are transmitted per second from the device to the gateway.The network takes into account the environmental conditions between the device and gateway to set the spreading factor for communication, which ranges from 7 to 12.A lower SF means more chirps per second, while a higher SF indicates fewer chirps per second [65].
PhySimulator evaluates the ability to receive two overlapping LoRa transmissions modulated with different SFs.The purpose of the PhySimulator authors is to study the SFs used by LoRa, which are nearly orthogonal, leading to collisions and incorrect reception in both close and distant conditions.The simulator is implemented in MATLAB and lacks a graphical interface.However, the user has the ability to modify parameters such as the number of load bits and bandwidth [36].

Comparison of Simulation Tools
In this section, we offer a comparative assessment of the leading simulation tools currently available.Previous works such as [35] and [36] focus on comparing features like simulation tool license type, programming language, and documentation.This work focuses on comparing the primary simulation tools along with their configurable parameters, as can be seen in Table 2.We extend [37] contributions by showing and discussing configurable parameters advantages and disadvantages.This information can be useful for researchers to select or contribute to a particular tool.
All the tools listed in Table 2 are open source.The parameter values in the table were obtained through code examples from each simulator and their values can be adjusted to suit the specific scenario being studied.However, if any changes are made to certain parameters such as device class or regional frequency-related parameters (such as frequency, bandwidth, transmission power, or packet size), it is necessary to ensure that the code meets the technical requirements of these parameters.On the other hand, modifications to other parameters such as the number of end devices (ED no.) and gateways (GW no.), coordinates of end devices and gateways, propagation models, and packet intervals (e.g., in order of seconds, minutes, hours) can be made without introducing new functions or classes to the code.The presented table illustrates that not all of the simulation tools support the Network Server feature.While modules II, III, and IV consider it an enhancement in the framework, others such as LoraSim and Phy Simulator do not aim to incorporate this feature as they focus on the physical layer.Regarding collision checking and ADR usage, nearly all tools provide the option to enable or disable these features.With respect to channel modeling, the majority of frameworks use the Log Distance channel propagation model.In contrast, Module I applies shadowing models such as Correlated Shadowing and Building Propagation Loss.The other modules (II, IV, Flora, LoraSim) employ the Log Distance model with its shadowing constant (Log Normal Shadowing).
The simulation tools mostly use the European frequency (868MHz), with the exception of Module I, which also offers the option to simulate with the North American frequency (915MHz).In terms of energy modeling, only a few tools (Module I, Module II, Flora, and LoraSim) take into account the power consumption of the LoRa Radio (such as the SX1272), while disregarding the consumption of other components such as the microcontroller and sensors.
As a result, it is clear that there is ample room for researchers to contribute to these open-source tools or even propose new simulation tools that can provide more comprehensive simulations.Such tools could include support for different LoRa frequencies and more sophisticated energy models that take into account all the relevant components in a LoRa system.

Channel Propagation Models
Electromagnetic wave propagation, commonly referred to as radio propagation, can be highly unpredictable in urban or forested areas due to the issues caused by the multipath phenomenon [66].In such situations, interference can lead to fading and result in the radio signal becoming too weak in some areas to be received accurately.In forested areas, multipath is primarily caused by reflections from vegetation, while in urban environments, reflections from buildings and vehicles are the primary cause [67].
Channel propagation models, also known as path loss models, describe the behavior of electromagnetic waves as they travel from a transmitter to a receiver in a wireless communication network [66].These models can be either empirical, deterministic, or a combination of both [68].Empirical models, unlike deterministic models, do not rely on terrain data such as elevation or building dimensions, which makes them less computationally intensive [69].These models are widely used in wireless network simulation tools, as they allow for the estimation and analysis of network coverage based on the effects of propagation parameters, such as the gateway antenna height and frequency [69].Therefore, to implement a successful network simulation, an accurate propagation model is crucial in establishing reliable communication links and estimating network coverage [15].

LoRaWAN Propagation Models
In a wireless network, the reduction in signal strength between a transmitter and a receiver is referred to as path loss, and it has a direct impact on the probability of packet delivery.Therefore, accurate prediction of path loss associated with a specific LoRa gateway position is crucial for optimizing network coverage and evaluating network performance [15].Channel propagation models are used to calculate this prediction.
The next sections review the most commonly used propagation models in LoRaWAN network simulations.

Log Distance Path Loss
The Log Distance Path Loss, also referred to as the Log Normal Shadowing or One Slope Model, is an extension of the Free Space model [70].While the Free Space model only considers an obstacle-free path between the transmitter and receiver, the Log Distance model is used to predict propagation loss in different environments, both indoor and outdoor, taking into account shadowing effects caused by objects such as trees, buildings, and hills [71].
The Log Distance model is a computationally efficient propagation model as it has few parameters to characterize the propagation [72].The main criterion of the Log Distance model is to consider path loss as a logarithmic function of distance [73].Therefore, the path loss (PL(d)) can be described by the following equation: Where PL(d0) is the path loss given a reference distance (commonly 1 meter), n is the path loss exponent, d is the distance between transmitter and receiver; and Xsigma is a zero-mean Gaussian distribution random variable (dB) with a standard deviation σ (dB).This variable is used only when there are shadowing effects in the study environment; otherwise, it assumes value 0 [71].
Another parameter that considerably affects propagation modeling performance is the path loss exponent n [71].It allows the Log Distance model to adapt to the environment characteristics.The occurrences of floors or walls in the modeled environment, for example, are expressed by it [70].Table 3 can be used as a reference for commonly used ranges of n values for different scenario types.Table 3. Common values of n for various scenarios [71].

Okumura-Hata
The Okumura-Hata model considers path losses due to shadowing effects such as diffraction, reflection, and scattering of the signal [74].It is applicable only for the frequency range between 150-1500 MHz and distances between 1-20 km.The Okumura-Hata prediction path loss in urban environments (PLurban), in dB, can be described by the equation:

 
69.55 26.16 log 13.82 log 44.9 6.55log log where fc is the frequency in MHz, ht is transmitter or base station height, ranging from 30 to 200 meters; hr is receiver or mobile antenna height, ranging from 1 to 10 meters; d is the distance between transmitter and receiver, and finally αh r is the correction factor for receiver antenna height [75].For small or medium-sized cities, it can be described by the equation: According to [74], real experiments shown that the path losses for suburban (PLsuburban) and rural (PLrural) environments, can be predicted based on the respective equations:

Building Penetration Loss
The Building Penetration Loss is responsible for calculating the path losses that are caused by buildings [76].The total loss calculation is based on three main components: losses due to external building walls (EWL), losses due to internal building walls (IWL) and gain in receiving power (GFH), proportional to the device's elevation height.Based on these components, the total path loss, in dB, caused by buildings (PLbuildings) for an indoor device is given by: The EWL component is modeled as a uniform random variable taking values over a certain range of intervals and probability, tabulated in [76].The range of values that EWL can take is intended to model different types of walls (e.g., building material, thickness) [77].The IWL, on the other hand, is calculated by means of the maximum between two values and can be expressed as: where Tor1 represents the loss due to the walls number and Tor2, represents the loss due to signal penetration into the wall given a uniform distance in the range 0 to 15 meters.Finally, GFH is responsible for expressing the gain in reception that an increase in the end device height produces.The detailed calculation of each of these components can be found in [76].

Correlated Shadowing
Shadowing correlation has an impact on different aspects of wireless networks, such as handover behavior and interference power [77].Consequently, when conducting network simulations, it is important to incorporate a propagation model that accounts for this correlation.The Correlated Shadowing model considers two types of correlation: firstly, when a transmitter communicates with two nearby devices, it is expected that these devices will experience similar shadowing effects.Secondly, when two transmitters are in close proximity and sending signals to the same node, the receiving device is likely to encounter correlated shadowing in the form of both signals being obstructed by a significant obstacle located between the transmitters and the receiver.A distancedependent exponential function describes the correlation between the shadowing experienced by nodes i and j as follows: where di,j is the distance between nodes i and j and d0 > 0 is an adjustable parameter called decorrelation distance, representing the distance at which the shadowing correlation is below the threshold e −1 and therefore shadowing can be considered uncorrelated.The Correlated Shadowing model generates 2D maps that describe the shadowing at various points within a specific region of space, given a transmitter's position.These maps are intended to represent the correlated shadowing effects that occur within the region [77].

Review of simulation parameters
The utilization of network simulation tools significantly enhances project testing by minimizing expenses, complications, and superfluous use of technologies.Moreover, these tools make it easier to evaluate the performance of different protocols.For effective simulation, precise parameters such as channel model, packet size, and number of nodes are necessary.Despite the transparency of their studies, many research papers that explore network simulations fail to address the matter of reproducibility by other researchers [78].
Ensuring the reproducibility of experiments is crucial for enhancing our understanding of research and serves as a guide for future studies, as well as a means of scientific recognition.It is a fundamental practice for generating new insights.This involves documenting and providing a clear understanding of all the steps and data involved in an experiment, with the aim of producing results that are equivalent or close to those presented by the original authors [79].
Simulation tools provide a wide range of configurable parameters and scenarios that can have a significant impact on performance.Therefore, it is important to address these parameters in the literature.In our study, we examined the literature on network parameters for LoRAWAN and used the findings to compile a list of parameters for review.We expanded upon the work of Gkotsiopoulos et al. [80], who categorized various performance determinants in the literature.Our analysis involved extending their list of parameters, focusing on both simulation and real-world experiments, and examining how the literature addresses each parameter.The list of parameters analyzed is presented in Table 4, with the additional parameters marked with an asterisk (*), indicating those not present in the previous work.
A reproducibility analysis of the literature on LoRaWAN networks was conducted, covering 17 papers published between 2016 and 2021.These papers addressed various topics, including the scalability of the technology, data monitoring, simulations with collisions, and others.The papers encompassed both real-world and simulated experiments, and were quantified based on the number of essential parameters they presented for reproducing the experiments in a simulation.The observed parameters included: 1. Confirmed type messages (ACK); 2. Signal transmission direction such as Uplink (UL), Downlink (DL) and Direct to Device (D2D); 3. Packet size; 4. Total duration of the experiment; 5. Packet interval; 6. Bandwidth; 7. Channel propagation model; 8. Transmission power; 9. Frequency; 10.Spreading factor; 11.Height of devices, gateways, or antennas; 12. Positions of nodes; and 13.Hardware platforms.
In the subsequent sections, we will explore the usage of these parameters in the literature, their importance, particularly in relation to reproducibility, and the commonalities found in the works that employ them.

Confirmation Messages
This parameter ensures that a packet is delivered by requiring the receiver to send an acknowledgment signal to the sender for the message received.The use of the ACK flag is common in simulation studies to analyze the scalability of LoRa technology.For instance, Bankov et al [81] and Farhad et al [96] evaluate the performance of the LoRaWAN protocol by increasing the density of devices and packets for a single gateway, but the latter also includes unconfirmed type messages and an urban scenario with buildings in their NS-3 simulations.In contrast, Abeele et al [58] consider various parameters along with confirmed and unconfirmed type messages to simulate different scenarios with one or more gateways.Some authors opt not to use this parameter in their simulations, particularly when experimenting with scenarios that require the fast delivery of critical packets.For example, Priyanta et al [90] simulate a port environment that demands high communication throughput, while Haiahem et al [88] evaluate network scalability performance for air pollution monitoring, which requires a report from each device at specified intervals.Additionally, Haxhibeqiri et al [85] completed its packet collision analysis without utilizing the parameter but may consider it in future work.Finally, Musek et al [86] aim to conserve battery power for its device by utilizing the unconfirmed message mode, and hence does not use the ACK flag.

Signal Direction
Uplink communication refers to the transmission of data from a node to an application, while downlink communication refers to the transmission of responses from an application to a node.Downlink communication can serve various purposes, such as notifying the device of a specific event or updating an application parameter.Downlink traffic can negatively impact the performance of the LoRa network, as the gateway cannot receive transmissions while it is sending data.However, Bankov et al [81] and Abeele et al [58] incorporate downlink transmission in their studies to investigate the performance of this parameter in the network.Bankov et al [81] reserve one of the four channels exclusively for the gateway to respond to the devices.On the other hand, Abeele et al [58] indicate that the limited downstream capacity significantly impairs the delivery rate of confirmed upstream message packets.Although increasing the density of the gateway can delay this effect, it cannot entirely eliminate it.

Packet Size
The packet size in LoRa is determined by the regional transmission parameters as well as the proximity to the nearest gateway.According to the LoRa Alliance [96], in the European scenario with a bandwidth of 125kHz, the maximum payload sizes are as follows: 51 bytes for SF10, SF11, and SF12; 115 bytes for SF9; and 242 bytes for SF7 and SF8.In addition, for a scenario with a bandwidth of 250kHz and SF7 the maximum payload is 242 bytes.
The payload sizes in the reviewed works varied between a minimum of 10 bytes and a maximum of 51 bytes, with the European frequency being the most commonly used in the experiments.Bankov et al [81], Ortiz et al [89], and Farhad et al [95], which tested scalability and packet reception mostly in urban and high-density scenarios, used a payload size of 51 bytes.Conversely, Masek et al [86] and Santos Filho et al [91], which focused on long-term and delivery reliability tests, respectively, used smaller payload sizes.

Duration of Experiments
In our survey, we are considering the duration of an experiment as the exact time spent on real and simulated tests.Based on the analysis of existing literature, it can be observed that there is no standardized approach for determining this parameter.Its selection and application vary based on the specific requirements and limitations outlined in each individual study.In the concrete experiments, as in articles [83] and [86], its duration is based on the characteristics of the collected elements: in the first, one of the experiments has a time of 1 week because it is an application on monitoring radioactivity containers, and this is the standard for garbage collection; in the second, the experiments have 2 days of data collection in the city of Brno, Czech Republic, and 2 months in the long-term test that explores the characteristics of the LoRaWAN channel in the same region.In articles [92] and [94], the total period is based on the experimental repetition method adopted by the authors.While in articles [58,88] and [95], the simulations last in proportion to the packet interval, monitoring periods and a short time of 600 seconds, respectively.

Packet Interval
We are considering the time duration between packet transmissions for each device.Research studies, such as [91,92] and [93], which focus on constant message transmission for monitoring purposes, have utilized packet intervals not exceeding 30 minutes.In other studies, such as those examining transmission reliability in specific scenarios, found in [58,81] and [90], different packet intervals were employed during experiments.Conversely, articles proposing projects aimed at conserving battery life, such as [83] and [86], feature longer packet sending intervals ranging from 1 hour or more.

Bandwidth
The LoraWAN protocol offers the option to use channels with a bandwidth of 125kHz, 250kHz, or 500kHz, which is dependent on the region or frequency plan.By increasing the bandwidth, the number of bytes transmitted in the same period is doubled.It is important to note that the design of antennas is influenced by the bandwidth and frequency used [97].In European experiments, as well as those conducted in Thailand [82] using a frequency of 433MHz and the European configuration, the default bandwidth is 125kHz.In contrast, articles from Brazil [84,89] enable higher bandwidth rates.The former investigates bandwidth intervals in different scenarios, including 500kHz in a forest, 125kHz and 250kHz in suburban and urban areas, and 125kHz for testing mobility in suburban environments.The latter maintains a bandwidth of 500kHz to examine the maximum communication ranges with and without mobility.

Channel Propagation Model
In Section 5, it is elucidated that channel propagation models play a crucial role in capturing the characteristics of electromagnetic waves within wireless communication networks.To provide comprehensive information, Table 5 presents a detailed overview of the propagation models employed in the surveyed papers.The majority of the papers examine urban scenarios with Line-of-Sight (LOS) and/or Non-Line-of-Sight (NLOS) conditions.The study environments mainly consist of Smart Cities or Smart Campuses.The Log Distance Path Loss model is the most commonly used propagation model in LoRaWAN network simulations, but few papers compare the coverage predictions of this model to actual field measurements, with the exception of [89].Some papers utilize more than one propagation model, as seen in [90,91], and [95].In addition to the Log Distance (LD) model, they include models such as Building Pen-etration Loss and Correlated Shadowing Propagation Loss that account for the shadowing effects caused by objects such as buildings and vehicles on electromagnetic waves.Out of the papers surveyed, only four of them [81,85,93], and [69], used the Okumura-Hata (OH) propagation model.However, only [85] and [69] validated this model in the field.Therefore, it can be concluded that validating the results of propagation model predictions and achieving accuracy in the analysis of network performance in terms of packet delivery is still a challenge in some research works that study LoRa coverage analysis through simulations, as comparison with real data is not always performed.

Transmission Power
In Europe, the maximum transmission power allowed for uplink transmission is 14dBm [98], and most of the surveyed works use this default value.However, in [86], the authors point out that this value was chosen to allow for longer communication ranges.In [87], Ortega et al. study an urban scenario in Colombia (using the US902-928MHz frequency band) and choose a transmission power value of 20dBm to avoid noise in the incoming signal.Another approach to transmission power can be found in [92], which uses Adaptive Data Rate (ADR) with a range of transmission power values between 2dBm and 14dBm in order to minimize energy consumption.

Frequency
The choice of allowable transmission power, bandwidth, and antenna radiation intensity in a LoRa network is determined by the regulations of the country or region, as well as the appropriate frequency plan.Hence, the frequency value is a crucial factor in experimentation.Further information on each frequency plan and corresponding countries can be found on the LoRa Alliance website.Among the reviewed articles, four different frequency values were used, including 433 MHz (EU433 in Asia), 868 MHz (European standard), 915 MHz (Latin America and Australia), and 923 MHz (AS923 in Asia).Interestingly, some experiments conducted in in the Americas [84] and Africa [92] used the 868 MHz frequency.
It is essential to consider the distinct characteristics of LoRaWAN supported frequencies.Comparing the two primary frequencies for LoRaWAN in Europe, 433 MHz and 868 MHz, as defined by the LoRa Alliance standard [97], reveals significant differences.The former requires a duty cycle (channel occupancy rate) of less than 10%, meaning that EU433 devices can only transmit data for 10% of the total time, with a maximum EIRP (Isotropic Effective Radiated Power) standard of 12 dBm.In contrast, for the EU868 frequency, the duty cycle is even stricter, with a lower limit of 1% per day, and its maximum EIRP is 16 dBm, allowing for greater transmission power.While the LoRaWAN AS923 frequency, used in Asia, shares characteristics with EU868, the AU915 frequency, utilized in Latin America and Australia, has no duty cycle limit, and its maximum EIRP is 30 dBm.

Spreading Factor Characteristics
The Spreading Factor is a crucial parameter in LoRa technology that controls the transmission speed.Smaller SFs result in faster transmissions with a higher data rate, while larger SFs lead to slower transmissions with a lower data rate.Since signals modulated with different spreading factors are orthogonal, there is no interference between them when sent simultaneously on the same channel frequency.Therefore, SF is an important parameter used in congestion control in a network, and it affects the data rate, time of a message in the air (ToA), battery expenditure, and receiver sensitivity.LoRa technology has a total of 6 spreading factors, ranging from SF7 to SF12 [99].
The selection of spreading factors is of great importance in determining the performance of a LoRa network.Transmissions with the same SF may experience intra-SF interference when they overlap in time, while transmissions with different SFs may suffer from inter-SF interference.Additionally, higher SF values can reduce network capacity due to longer data transfer times, whereas multiple SFs can increase capacity significantly due to parallel transmissions that are mostly orthogonal to each other.
In the reviewed papers, the SF values in the devices are adapted to the specific problem being investigated.Smaller SFs are used to reduce battery consumption, higher SFs are used to achieve long ranges, and varying intervals are used to find the best performance.
Van den Abeele et al. [58] used an error model to test three types of allocation: random, fixed (same SF for all devices), and PER (smallest SF whose error rate per packet falls below a threshold for each device).PER had the best results and was used in the final design.
Santos Filho et al. [91] conducted a performance evaluation by testing three different strategies to simulate an industrial scenario, involving nodes with varying priority levels: higher (alarms) and lower (regular) priority.The first strategy, referred to as the basic strategy, assigns the lowest SF based on the received power at the receiver, with a transmit power of 14 dBm.The second strategy, known as the shift strategy, allocates each node to the SF one level higher than the one assigned in the basic strategy.However, the authors found the SF reservation strategy, where alarm nodes are assigned to the highest SF among them in the basic strategy while regular nodes cannot be assigned to this specific SF, to be an inadequate solution.
In the street lighting system, presented in [92], the Adaptive Data Rate (ADR) based on Signal-to-Noise Ratio (SNR) is used.For a large number of gateways, devices use smaller SF, which reduces power consumption and extends network lifetime.
Mnguni et al. [94] present gateway positioning algorithms that assign the SF after simulating the network five times and calculating the signal strength between the device and gateway.

Gateway Position and Elevation
The placement of gateways also impacts the performance of LoRaWAN experiments.The literature suggests that locations with fewer obstacles, lower signal penetration loss, and support for multiple SFs, especially those with lower ToA values, have shown better results.Several studies, such as [69,83,85,87,90,94], and [95], have been conducted in urban areas, where gateway heights are chosen to mitigate signal interference.The gateways are typically placed on top of or above buildings, with minimum and maximum heights of 15 m and 50 m, respectively.These limits have been used in [95], which demonstrated that increasing the antenna height significantly improves the success rate of receiving packets.
In other studies, such as [58,91], and [93], the placement of gateways is determined differently.In the first two studies, the gateways are placed inside a circular area with the devices, and their coordinates are fixed based on the number of simulated gateways.In [93], the height of the gateways varies from 30 m to 200 m above the ground level, according to the limits of the propagation model used.
In contrast, [82] sets the same height for the device and the gateway in the first experiment to test the rate of lost packets by changing the distance.

Node Distribution
The distribution of nodes across a geographical area is another important factor in evaluating the performance of a LoRaWAN network.The number and location of gateways used in the experiment have a significant impact on the network's ability to transmit and receive packets.Overcrowding in a particular area can decrease the network's performance, while placing nodes near gateways can improve it.
In experiments that simulate scenarios with a single gateway, such as those in [81,85,88,90,94,95], the performance is tested with a varying number of devices, including up to 10,000 devices in some cases, which can overcrowd the area and negatively affect the performance of the network.
On the other hand, experiments in [82,83,84,86,87,89] and [93], which are either real experiments or simulations based on existing telemetry systems, use a smaller number of nodes fixed in specific positions to measure targeted metrics.
In some studies, such as [58,91], and [92], multiple gateways are used, and their positions are varied to evaluate network performance.Increasing the gateway density can mitigate the negative impact of overcrowding, but cannot completely eliminate it.

Hardware Platforms
Real-world experiments were conducted to evaluate the performance of LoRaWAN Networks in papers such as [82,83,84], and [86].These experiments utilized different types of hardware, showcasing a wide range of options for end devices, LoRa radio modules, and gateways.Table 6 presents a description of the primary hardware used in the evaluation of LoRaWAN Networks.In [82] and [84], the Arduino Uno microcontroller was chosen.In [83] and [86], the ATSAML21G18B micro-controller and a board developed by the authors themselves with the built-in Microchip RN2483 LoRa radio were utilized, respectively.Regarding other radio modules, the works [82,83], and [84] employed the Libelium LoRa module (433MHz), RFM95W-868S2, and Dragino RF96, respectively.The gateway used in [82] and [83] consisted of the NodeMCUV2 microcontroller and the Libelium LoRa radio module, while the MultiConnect Conduit model MTCDT-210A was used.In [84], Dragino RF96 kits were used for device-todevice communication, and B-L072Z-LRWAN1 LoRA/Sigfox kits with an onboard Semtech SX1276 transceiver chip were used for device-to-gateway communication.

Conclusion and Future Research Directions
This survey provides a comprehensive overview of LoRa and LoRaWAN technologies, with a particular focus on simulation tools used in research papers and the various parameters employed for simulating these networks.The survey also discusses challenges and opportunities that exist within this research area.
A key contribution of this work lies in the detailed and critical analysis of the significance of LoRaWAN parameters in the existing literature.To achieve this, both simulation-based and real-world studies were reviewed, elucidating the key characteristics of the technology as described by these works.This analysis offers valuable insights into how authors approach LoRaWAN experiments and their perspectives on the essential parameters for future studies to reproduce.
In Section 4, it was demonstrated that while several options for LoRa and LoRaWAN simulation tools already exist, the majority of them primarily focus on reproducing European (868 MHz) simulation scenarios, neglecting implementations for other regions.Additionally, the absence of complex energy models in these tools was observed, as most of them only consider the energy consumption of the LoRa Radio, disregarding the microcontroller and sensor consumption.
Another challenge arises from the limitations of certain tools in validating predicted propagation models in LoRa networks using real measurements, which compromises the accuracy of network performance assessment, particularly in terms of packet delivery.Furthermore, most of the available simulation tools lack robust class and parameter settings for the Network Server.Consequently, there are ample opportunities for contributing to the improvement of these simulation tools by addressing these limitations.

Figure 1 .
Figure 1.Structure of this survey.

Table 1 .
Comparison of popular LPWAN technologies

Table 2 .
Simulation tools with default main parameters.

Table 4 .
Parameters that are explored in the literature.

Table 5 .
Propagation model details

Table 6 .
Hardware description of LoraWAN work using real world experiment.