IEEE REVISTA IBEROAMERICANA DE TECNOLOGIAS DEL APRENDIZAJE, VOL. 11, NO.
1, FEBRUARY 2016 33
Framework to Evaluate M-Learning Systems:
A Technological and Pedagogical Approach
Christian X. Navarro, Ana I. Molina, Miguel A. Redondo, and Reyes Juárez-Ramírez
Abstract— This paper presents the analysis of recent research
on mobile learning and usability areas, applying a systematic
mapping study. The aim is to understand the tendencies and
needs in the m-learning field. The results demonstrate that
research in the area has grown significantly since 2013, and we
identify a necessity when we see that not all the m-learning
applications have used usability tests; we also did not find
guidelines or frameworks to evaluate them. With these results and
tendencies, we propose an evaluation framework for m-learning
applications, considering pedagogical usability and user interface
usability, to improve the quality of m-learning applications.
Index Terms— m-learning, mobile learning, usability, mapping
study, evaluation.
I. I NTRODUCTION
N OWADAYS, the use of mobile devices is widespread, in
our daily life, and in different areas such as culture and
education. These devices have achieved historical access to
communication and information.
The “Ericsson Mobility Report of 2013” predicted the
growth of mobile smartphones subscriptions to be ten times
higher between 2013 and 2019. Based on this prediction we
believe that by 2019 there will be 5,600 billion subscrip-
tions [1].
These devices are easy, flexible and constantly improving,
which allows us to believe that there are many new possibilities
for users to benefit from, and one of these is the educational
area. Mobile devices are also useful tools that go everywhere
with us; this could be perfect for any student during the
learning process. The information is available at any time
or place. UNESCO has even considered mobile devices as
a platform to allow equal opportunities in education among
different countries. Additionally, the publication of papers
related to m-learning has increased over the last few years [2].
Manuscript received August 31, 2015; revised December 16, 2015; accepted
December 17, 2015. Date of publication March 2, 2016; date of cur-
rent version March 3, 2016. This work was supported in part by the
Project EDUCA-Prog through the Ministry of Sciences and Innovation under
Grant TIN2011-29542-C02-01 and Grant TIN2011-29542-C02-02, in part by
the Project from the Group of Communities of Castilla-La Mancha, InterGroup
under Grant PPII-2014-021-P, and in part by the CYTED Project under
Grant Red 513RT0481.
C. X. Navarro is with the Faculty of Engineering, Architecture and Design,
Autonomous University of Baja California, Ensenada 22860, Mexico (e-mail:
cnavarro@uabc.edu.mx).
A. I. Molina and M. A. Redondo are with the College of Computer
Science, University of Castilla–La Mancha, Ciudad Real 13071, Spain
(e-mail: anaisabel.molina@uclm.es; miguel.redondo@uclm.es).
R. Juárez-Ramírez is with the Faculty of Chemical Science and Engineering,
Autonomous University of Baja California, Tijuana 22390, Mexico (e-mail:
reyesjua@uabc.edu.mx).
Color versions of one or more of the figures in this paper are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/RITA.2016.2518459
1932-8540 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
However, mobile devices present a challenge due to the
size of their screens and the amount of information that
can be displayed. This poses a problem in the area, related
to the usability, because of the difficulties that could be
generated during the learning process. For these reasons, this
paper presents the recent situation of research in the area of
usability on m-learning. We identify the different approaches
of research papers in the area of m-learning, as well as the
most commonly used mobile devices and operating systems.
The main goal is to understand the tendencies and opportunity
areas within m-learning, through a systematic mapping study.
Moreover, we propose the creation of an evaluation framework
for m-learning applications, considering pedagogical usability
and user interface usability, in order to improve the quality
of use, and the students’ experience in the mobile learning
environments [3].
The organization of this paper is as follows: Section II
presents the new tendencies of education and usability in
m-learning. Section III shows us the results of a literature
mapping study in the area of usability in m-learning.
Section IV presents a proposal of an evaluation framework for
m-learning systems. Finally, section V outlines the conclusions
of this work.
II. N EW T RENDS IN E DUCATION
The NMC Horizon Report identifies which new technologies
have higher potential in the educational areas [4]. These
tendencies are already being applied in different innova-
tive institutions around the world. One of these trends is
MOOCs (Massively Open Online Courses). Another technol-
ogy which now has a large acceptance among mobile users
is tablets. Their main advantages are connectivity, multifunc-
tionality and portability. For these reasons, they are proving
to be a useful way of supporting education, without limits
in time and space, allowing access to educational material,
and representing a tool for teachers to use in the learning
process [5].
The game-based learning is another tendency quoted in
the NMC Horizon Report that will continue to grow in the
following two or three years [4]. The predictions in the
analysis by TechNavio related to the global market on
game-based learning are for an increase of a 15.5% in the
period of 2012 to 2016 [6]. And one of the factors contributing
to the growth is the mobile learning games.
Learning analytics is also a tendency, which is related to
the interpretation of the data generated by students when they
interact with a learning context. Recent studies show that this
34 IEEE REVISTA IBEROAMERICANA DE TECNOLOGIAS DEL APRENDIZAJE, VOL. 11, NO. 1, FEBRUARY 2016
will guide us to the creation of personalized environments
for learning, adapting the students’ responses throughout their
interaction with information [5].
A. M-Learning in Education
The definition of M-learning has evolved over the last few
years, in a way that many authors have made different defini-
tions. For example, Quinn [7] defines it as “a type of e-learning
through mobile devices”. O’Malley [8] said “learning that
takes place when the student benefits from the opportunities
provided by mobile technologies”. Sharples’ work [9] changed
the way of thinking about m-learning, focusing on the student.
According to this author, the student has mobility and not
the technology. Students pick the technology on hand while
they move between different contexts, including cell phones,
computer, books and notepads. And in 2013, Crompton [10]
defined it as “learning across multiple contexts, through social
and content interactions, using personal electronic devices.”
This last definition is focused on the students and their learning
process.
During 2012 and 2013 UNESCO published several docu-
ments about m-learning, to achieve a better understanding of
how mobile technology can improve equal access to education
in different countries [11].
M-learning has been studied in different contexts of educa-
tion, such as formal and informal education, and continuous
learning. There is a known division between formal education,
which happens inside a classroom and informal which happens
in a house or in different contexts outside of a classroom.
M-learning could help reduce the separation between these
types of learning.
Inside formal education there are two popular types of
mobile learning models in schools: one is program 1 to 1,
where each student is provided with a device, and the other
type is BYOTs (Bring Your Own Technology), where most
students have their own devices and they bring them to
class [12].
Also, within informal learning we can consider permanent
or continuous learning, which means learning that happens
along with life. This does not change one’s official cur-
riculum, but it evolves with life, depending on the personal
and professional situation. Permanent learning can happen
in any place and its main characteristics are integration and
innovation [13].
B. Usability in Mobile Applications
Mobile usability can be considered a specialty in the
usability field that is currently emerging. Researchers in the
Human Computer Interaction field found that to produce
computer systems with the correct usability, it is important
to understand psychological, ergonomics, organization and the
social factors that define how people work [14].
Jakob Nielsen has studied usability since 1993. He is the
founder of “discount usability engineering” which outlines
the use of effective methods to improve the quality of user
interface design, and has applied his methods in the area of
mobile devices through empirical research published in his
book Mobile Usability [15]. This author explains usability in
terms of the general acceptance of a system that includes social
acceptability, and practical aspects such as reliability, cost,
compatibility and usefulness [16]. In 2012 he defined usability:
“a quality attribute that assesses how easy user interfaces are
to use”. The word usability also refers to the methods to
improve the ease of use during a design process [17].
The most commonly used techniques to measure usability
are heuristic evaluations; however, they are not yet adapted
to the characteristics in mobile computing, which means that
they do not consider the context of use and mobility. It is
necessary to adapt the methods for these topics to reach the
proper importance within the mobile usability tests [18].
III. M APPING S TUDY
Considering the facts stated in this document, we can
see that usability is important when we talk about mobile
computing and in general, as well as when we approach it
in an m-learning educational context. For this reason, part of
this work focuses on researching existing literature on this
topic, using a systematic mapping study. This methodology
will allow us to evaluate and interpret all the papers relevant
to mobile usability and m-learning [19], emphasizing mobile
devices such as smartphones and tablets.
The activities developed during this work, according to
the methodology, are described in the following subsections.
We should mention that the data presented in this study is
an update of the mapping study presented in [20], which
considered papers published up to March 2013.
A. Research Questions
Because our main interest is to know the current status of
research on m-learning and mobile usability areas, we have
outlined the following research questions:
1) What is the current state of m-learning and mobile
usability research? (Considering only papers using mobile
devices such as smartphones and tablets). Is there an increase
in the number of publications over the last few years?
2) What are the different approaches for publications in
the areas of m-learning and usability? What kind of mobile
devices and operating systems are the most commonly used
in such papers?
In order to respond to these questions the key words
used in the search were: “mobile learning”, “m-learning”
and “usability.” Therefore the search string was: (“mobile
learning” OR “m-learning”) AND “usability”, which
had some variations depending on the data base being
searched.
B. Identifying the Appropriate Databases
The research papers included in this study were found by
searching through the available literature in the databases
IEEE Digital Library, Science Direct On Site, ACM Digital
Library, and Scopus. The string search mentioned in the
previous subsection was performed in titles, abstracts, and key
words of the documents. We included all the works related to
m-learning and mobile usability, specifically those that use
NAVARRO et al.: FRAMEWORK TO EVALUATE M-LEARNING SYSTEMS
Fig. 1. Total of papers found during the initial search, in the different
databases.
Fig. 2. Number of relevant papers extracted from each database.
Fig. 3. Selected papers according to their publication year.
mobile devices such as smartphones and tablets, published in
English, parts of journals, conferences or workshops.
C. Results
The first search produced a total of 1546 publications
(Fig. 1). We analyzed all the titles and abstracts obtained from
each database, and if there were doubts about the relevance
of a paper we also read the introduction and/or conclusions
to establish if it would be included or excluded from this
research.
168 relevant documents remained after considering the
inclusion/exclusion criteria, and going through both of the
search phases (Fig. 2).
Figure 3 shows us the resulting number of articles published
from 2006 to 2014. We can see that research in this area
has increased since 2010, and by the year 2013 it had
35
Fig. 4. Approaches according to the research purpose.
Fig. 5. Types of mobile devices used in m-learning.
Fig. 6. Applications according to the academic level of the audience.
significantly grown. The search was performed by the end
of November 2014, so we should also consider that the last
months of the year will bring the numbers up, and in view
of the boom of mobile devices in education, we believe that
research in the field will continue to grow in the following
years.
The 168 relevant articles show different approaches.
According to the purpose of their research, we have
classified for a better interpretation: m-learning apps,
guidelines/frameworks, specific aspects of m-learning, study
cases, and analysis and tendencies of m-learning (Fig. 4).
Among the 61 articles that focused on m-learning, different
devices were used (Figure 5). In these apps, 69% have shown
to be native apps, which means they were stored in the mobile
device, while the other 31% are web-based apps.
Fig. 6 shows the classification of the applications according
to the academic level of the audience. Furthermore, we found
that 40% of these apps are game based.
36 IEEE REVISTA IBEROAMERICANA DE TECNOLOGIAS DEL APRENDIZAJE, VOL. 11, NO. 1, FEBRUARY 2016
Fig. 7. Classification sequence of the analyzed papers.
Only 52% of the considered applications have performed
usability tests, and 44% of these had usability tests based on
heuristic evaluations or expert evaluations.
There are 42 articles about guidelines and frameworks in
the area of m-learning or mobile usability (Fig. 4), and out
of these, 28 are development frameworks and 14 are for
evaluation.
Out of the 31 articles that approach specific aspects of
m-learning (Fig. 4), all of them treat different topics, for
example: the factors that provoke students to use smartphones
in an educational environment, how to customize an
m-learning application, or design proposals for education
interfaces.
Among the 22 papers about case studies, there are different
topics, mostly related to the effectiveness of using m-learning
in different disciplines, and other papers about the perceptions
and teacher’s beliefs about m-learning.
In the 12 articles centered on the analysis and tendencies
of m-learning (Fig. 4), we have two systematic studies, and
others related to the challenges and opportunities in the
m-learning environments.
D. Analysis and Discussions
In order to represent a full description of the classification
of articles within this research, we present a scheme with the
different categories of the analyzed papers (Figure 7).
Because of the focus of this research, the articles considered
in the following analysis are those that consider usability
in their guidelines or frameworks to develop or to evaluate
m-learning environments.
There are 18 projects about development guidelines and
frameworks that consider usability and that are oriented to
the development (Figure 7, classification 4).
Also, there are evaluation guidelines and frameworks
that consider usability, but we only found two articles
where frameworks are developed to evaluate mobile devices
(handhelds) related to m-learning. In the first paper the author,
after identifying the strengths and weaknesses in the devices,
makes suggestions in terms of the technical specifications
Fig. 8. Conceptual framework for evaluation of m-learning system.
Fig. 9. Evaluation categories and elements of pedagogical usability.
required for m-learning. The second one presents a rubric to
evaluate the quality of the mobile applications to learn English
as a second language.
With this information, we can see that there is a lack of
evaluation tools for m-learning applications. In the following
section we propose a framework that will be the basis for
developing an evaluation tool for m-learning applications,
considering pedagogical usability, and user interface design
usability.
IV. P ROPOSAL FOR AN E VALUATION F RAMEWORK
FOR M -L EARNING S YSTEMS
In [20] we propose an evaluation framework for m-learning
systems, but this has suffered some changes since the last
publication. These changes have been motivated by the feed-
back of different experts in the usability area, with the aim to
eliminate confusions that the previous classification may have
caused.
The proposed evaluation framework is divided in the first
level into two categories, pedagogical usability and user
interface usability as shown in Figure 8, and then these are
subdivided. Following these graphics we describe each one of
the elements of the evaluation framework with the factors and
subdivisions. (Fig. 9 and Fig. 10).
A. Pedagogical Usability
Mobile learning environments must address educational,
pedagogical and usability factors in order to ease and support
teaching and learning activities. These factors will provide
an appropriate environment for the educational practices.
Pedagogical usability in this framework has five subdivisions;
NAVARRO et al.: FRAMEWORK TO EVALUATE M-LEARNING SYSTEMS
Fig. 10. Evaluation categories and elements of user interface usability.
also each one of these has different criteria, which will now
be explained in detail.
1) Content: In an m-learning environment the content must
consider the pedagogical aspects to generate efficient learning.
These aspects correspond to the following criteria.
Organization: The content should be organized in manage-
able modules, or other types of units. Sequencing of these
modules is important; the key topics should have priority.
Objectives: These must be defined at the beginning of the
didactic sequence, formulated in simple and precise language.
Students should be able to understand what they are expected
to know after they use the m-learning application, in order to
increase engagement.
Previous Requirements: If necessary the m-learning appli-
cation should inform the user of required previous abilities
or knowledge. It could provide links to previous materials,
in order to allow students the option of returning and reviewing
previous information.
Language: Language must be simple, clear and appropriate
for purpose and audience.
Reliability: Information must be correct, updated and error
free.
Cognitive load: Contents should be broken into appropriate
chunks, for students to be able to process them without too
much cognitive load. The amount of new key concepts should
be limited to one or two, depending on the complexity.
Relevance: The materials should be based on the students’
characteristics, interests and motivations and must be useful
for their future.
Complexity: Learning materials should challenge the
student with a difficulty level according to their knowledge
and academic level.
Time Limits: Mobile learning should present a sequence
of short instructional moments between 30 seconds and
10 minutes [21].
Resources: The m-learning application provides access to a
range of external resources, appropriate to the learning context,
including links to the external World Wide Web. These links
must be kept up-to-date and the information should be adapted
for mobile devices.
2) Multimedia: M-learning applications should support
different types of media, such as video, audio, text or anima-
tions, all of them with high quality and appropriate for mobile
devices. In the following section we will explain the criteria
defining it.
37
Aligning With Objectives: The multimedia contents should
have a strong connection to the objectives. Animations, images
and sounds must help students with their learning objectives.
Adequacy: The multimedia materials should be presented in
a format that makes it easier to learn concepts.
Time Length: Length for multimedia animation, video and
sound should be between 2 and 5 minutes.
Quality: Multimedia should have sound fidelity, quality
video, images must be attractive, and the size of the files
should be appropriate for the device.
Balance: There should be the right proportion of media
objects at the appropriate positions in the content. Materials
must represent multiple views of knowledge and tasks.
3) Tasks or Activities: The task or activities in a mobile
learning environment are an important part of the learn-
ing process. We will now define its criteria for proper
consideration.
Aligning With Objectives: Tasks or activities must have a
strong connection to the objectives.
Sequencing: Tasks must allow students to integrate new
information with prior learning to generate knowledge.
Problem-Based Learning: Tasks should require students
to compare and classify information, make deductions, and
promote creativity.
Authenticity: The task should reflect real-world practice, rel-
evant to professional practice, generating interest and engage-
ment in students. They must support transference of skills
beyond the learning environment and critical thinking.
Interactivity: Tasks should engage students in problems to
solve, that take advantage of state of the art mobile design
(field investigations, taking pictures, videos, augmented reality,
QR codes).
Adequacy: Tasks should be congruent with the content and
capabilities of the target audience.
Self-Evaluation: Software should allow opportunities
wherever appropriate for self-assessment that advance
students’ achievement.
4) Social Interaction: Socialization is fundamental for the
learning process, and m-learning environments must promote
it among the students. In order to assess this dimension,
we will explain the following criteria.
Dialogue: The m-learning application should allow students
to communicate with their classmates or teachers (chat, notice
board or social networks).
Collaboration: The mobile learning environment should
allow students to do group work with their classmates.
Discussion: The mobile learning environment should
provide opportunities to support learning through interac-
tion, discussion and other collaborative activities. Discussion
requires students to participate by posting their understanding,
in order to receive feedback and allow students to evaluate and
learn from other groups or students through their uploaded
works and opinions.
Sharing: The m-learning application should allow students
to share photos, videos or any other documents related to their
work via social networks such as Facebook or Twitter.
5) Personalization: Personalization gives the students the
freedom to control different options related to their learning
38 IEEE REVISTA IBEROAMERICANA DE TECNOLOGIAS DEL APRENDIZAJE, VOL. 11, NO. 1, FEBRUARY 2016
style or with the sequence of work. In order to assess person-
alization we will consider the following criteria.
User Control: Students should have the freedom to direct
their learning, either individually or collaboratively, and have
a sense of ownership of it. They can also have control of
the sequence they follow, through optional routes for their
progress.
Individualization: Students can customize their learning to
suit their personal learning strategies (information can be
reviewed in videos, text, or audio files).
B. User Interface Usability
The user interface usability is one of the strengths for mobile
applications acceptance by clients. In the context of mobile
learning environments, the interface should be easy to learn,
easy to be recognized and remembered by users. In this frame-
work we have identified five subdivisions that are also divided
into different criteria, which will now be explained in detail.
1) Design: A good design is very important to improve
usability in a user interface. In order to assess this aspect we
will consider the following criteria.
Attractiveness: Design should be esthetic and attractive
to the students. The look must be pleasant, and the screen
interface must not contain information that is irrelevant or
rarely needed in the system. The user interface should not be
overloaded with information, or include colors that negatively
affect the visual perception of the contents.
Presentation: The user interface should follow good
presentation guidelines, with respect to organization and
layout. The instructions on how to use the mobile application
must be visible or easily retrievable when appropriate. The
information must be organized in a logical way, and pages
should correspond or adapt to the screen size.
Consistency: The design interface should be consistent and
logical throughout the overall design. It must follow standard
conventions, with a consistent selection of style for text,
themes and buttons.
Understandability: The used terms, phrases, symbols and
concepts are written in everyday language and must be intu-
itive for students to be able to easily understand the purposes
of the system functions.
2) Navigation: Navigation must be simple for the users,
always letting them know where they are, where they have
been and where they can go. This could be assessed with the
following criteria.
Ease of Use: Navigation should provide functions which
are easy to understand, remember and use. Options should
have the correct visibility. No more than three clicks should
be required to get from the initial page to the content.
Orientation: This will allow the student to clearly identify
where they are, how to get back to the main menu and easily
navigate through the system. Each screen should display its
title.
Freedom: When a student chooses a system function by
mistake, the system should always allow the user to go back
to previous windows. Allowing options such as undo and redo
are important to maintain design freedom.
Search: The application must provide mechanisms to help
users find content.
3) Customization: Customization means to modify some-
thing according to personal preferences. This is important
because it allows the students to change the design or the
navigation on their devices. We assess this using the following
criteria.
Flexibility: Design should have flexibility by allowing stu-
dents to change items in the interface, and also provide
shortcuts, and accelerators speeding up the interaction by
tailoring frequent actions. The flexibility in design allows
different levels of users, from novice to expert.
Input/Output Support: This element is of special interest
in the mobile context. In fact, one of the main limitations
in the mobile context is interaction. Therefore, m-learning
applications should allow different input and output options
in order to improve student accessibility.
Adaptability: Ability of a mobile learning environment to
be adapted to different devices, without the need to take
further actions (configurations) and without super-imposition
of objects or any loss of information.
4) Feedback: The application should help the students con-
solidate their knowledge and reinforce new skills, during their
interaction through their learning process. We will now explain
the criteria that define it.
Progress: The application should present the level of
achievement within the whole program and in each unit.
Encouragement: Students should be given constructive feed-
back when they have made a significant advance, in order to
encourage them and generate trust.
Support: The application should provide support for the
students to achieve their learning goals. If the student makes a
mistake when performing a task, the application should offer
opportunities to get to the right answer, or at least provide
an explanation. There should also be opportunities to access
extended feedback from instructors, experts, peers or others,
allowing the right kind of support when needed.
Precise: The feedback provided should be appropriate to
the content, problem or task, and immediate to the activities.
System Status: The system should allow the students to
access general information such as battery, network status,
or the time.
Alerts: The system should provide feedback though alerts,
by reminding deadlines, notifying of events or activities.
Error Prevention: When the user selects two options at
the same time by mistake, the application should request a
confirmation, and zoom in on the options in order to make
the manual selection easier.
5) Motivation: The motivation should increase the students’
attention in the learning process through different ludic and
pedagogic strategies, applied to the task dynamics. In order to
asses it we consider the following criteria.
Game-Based Learning: When a student learns through
games, they have increased motivation and interest through
the role of fun in learning.
Competitiveness: The m-learning application should be
able to represent and allow competitiveness between stu-
dents through opportunities to work at the same time
NAVARRO et al.: FRAMEWORK TO EVALUATE M-LEARNING SYSTEMS
on a task, or showing leader boards with achievement
scores.
Convenience: Students should find it more useful to learn
provided topics with mobile learning than with conventional
methods, or in a classroom. The m-learning environment
should provide a quick and easy form to learn a new topic
or review previous topics.
Badges: The application should generate symbolic badges
or rewards when the student has made a significant
achievement.
C. Evaluation Tool
In view of this criteria we have considered it appropriate
to develop a web tool that will allow us to test (through
a questionnaire) the m-learning application characteristics,
related to the support of the different dimensions or subdi-
visions (or criteria) included in the proposal of a conceptual
framework, that has been previously described. This tool
will be used to evaluate the m-learning application, to score,
compare and detect opportunities of improvement, related to
the characteristics in the framework.
V. C ONCLUSION
This state of the art analysis has allowed us to have a gen-
eral vision about m-learning, the tendencies and inclusion in
education. The results of the systematic mapping study shows
the number of papers in the area have increased significantly
since 2013, and due to the boom of mobile devices in
education, we believe that the publications on the topic will
continue to increase in the following years. We can also see
that only 52% of the analyzed applications used a usability
test, and only 44% are based on heuristic evaluations or
revised by experts.
Based on these results, we propose the creation of an
evaluation framework for m-learning applications, considering
pedagogical usability and user interface usability, in order
to improve their overall quality in use. We know that
m-learning applications are up-to-date available options to
assist the students in the learning process, therefore our
framework will allow for the evaluation of the application’s
quality to improve the learning experience for the students.
ACKNOWLEDGEMENTS
C. X. Navarro expresses his gratitude to the Program of
teacher development included in the Direction for Academic
Improvement (DSA) from SEP (Secretary of Public Education
in México), for the scholarship given to perform his
PhD studies.
R EFERENCES
[1] Ericsson Mobility Report: On the Pulse of the Networked Society,
Ericsson, Stockholm, Sweden, Nov. 2013.
[2] C. X. Navarro, A. I. Molina, and M. A. Redondo, “Marco para la
evaluación de sistemas m-learning: Análisis de la situación y propuesta,”
in Proc. 15th Int. Conf. Human Comput. Interact. (INTERACCIÓN),
Puerto de la Cruz, Spain, 2014, pp. 82–89.
[3] J. C. S. Prieto, S. O. Migueláñez, and F. J. García-Peñalvo,
“Understanding mobile learning: Devices, pedagogical Implications and
research lines, Edu. in the Knowl. Soc. (EKS), vol. 15, no. 1, pp. 20–42,
2014.
39
[4] L. Johnson, S. A. Becker, M. Cummins, V. Estrada, A. Freeman, and
H. Ludgate, NMC Horizon Report: Higher Education Edition. Austin,
TX, USA: The New Media Consortium. 2013.
[5] E. Durall, B. Gros, M. Maina, L. Johnson, and Y. S. Adams, Perspectivas
Tecnológicas: Educación Superior en Iberoamérica 2012-2017. Austin,
TX, USA: The New Media Consortium, 2012.
[6] TechNavio. (Mar. 2013). Global Game-Based Learning
Market 2012-2017, (Ref. IRTNTR2043). [Online]. Available:
http://www.studymode.com/, accessed Mar. 2013.
[7] C. Quinn, “mLearning: Mobile, wireless, in-your-pocket learning,” LiNE
Zine, Staunton, VA, USA, Tech. Rep., 2000.
[8] C. O’Malley et al., “Guidelines for learning/teaching/tutoring in a mobile
environment,” Public deliverable from the MOBILearn Project (D.4.1),
2005.
[9] M. Sharples, J. Taylor, and G. Vavoula, “Towards a theory of mobile
learning,” Proc. mLearn, vol. 1, no. 1, pp. 1–9, 2005.
[10] H. Crompton, “A historical overview of mobile learning: Toward learner-
centered education,” in Handbook of Mobile Learning, Z. L. Berge and
L. Y. Muilenburg, Eds. London, U.K.: Routledge, 2013.
[11] C. Shuler, N. Winters, and M. West, The Future of Mobile Learning
Report: Implications for Policymakers and Planners. Paris, France:
UNESCO, 2013.
[12] Cisco, University Embraces Bring-Your-Own-Device With Wireless Net-
work, Customer Case Study Cisco Syst., Inc., San Jose, CA, USA, 2012.
[13] C. C. Valero, M. R. Redondo, and A. S. Palacín, “Tendencias Actuales
en el Uso de Dispositivos Móviles en Educación,” Edu. Portal Amer.-
Dept. Human Develop., Edu. Culture, La Educación Digital Magazine,
Tech. Rep. 147, 2012, pp. 1–21.
[14] A. Kukulska-Hulme, “Mobile usability in educational contexts: What
have we learnt?” Int. Rev. Res. Open Distance Learn., vol. 8, no. 2,
pp. 1–16, 2007.
[15] J. Nielsen and R. Budiu, Usabilidad en Dispositivos Móviles. Madrid,
Spain: Ediciones Anaya Multimedia, 2013.
[16] J. Nielsen, Usability Engineering. San Francisco, CA, USA: Morgan
Kaufmann Publishers Inc., 1993.
[17] J. Nielsen, Usability 101: Introduction to Usability. Online Article from
Nielsen Norman group, Jan. 2012.
[18] C. Cuadrat, M. S. Veloso, and J. J. R. Soler, “Towards the validation
of a method for quantitative mobile usability testing based on desktop
eyetracking,” in Proc. 13th Int. Conf. Interac. Persona-Ordenador, 2012,
p. 49.
[19] B. Kitchenham and S. Charters, “Guidelines for performing systematic
literature reviews in software engineering,” Keele Univ., Keele, U.K.,
Tech. Rep. EBSE-2007-01, 2007.
[20] C. X. N. Cota, A. I. M. Díaz, and M. Á. R. Duque, “Develop-
ing a framework to evaluate usability in m-learning systems: Map-
ping study and proposal,” in Proc. Technol. Ecosyst. Enhancing
Multiculturality (TEEM), Salamanca, Spain, 2014, pp. 357–364.
[21] Z. Hu, “Emerging vocabulary learning: From a perspective of activities
facilitated by mobile devices,” in Proc. World Conf. E-Learn. Corp.,
Government, Healthcare, Higher Edu., Chesapeake, VA, USA, 2011,
pp. 1334–1340.
Christian X. Navarro received the master’s degree
in computer science from the Center of Scien-
tific Research and Superior Education in Ensenada,
in 2002. He joined the Faculty of Engineering,
Architecture and Design, Autonomous University of
Baja California, Mexico, as a full-time Professor.
He is currently pursuing the Ph.D. degree with the
University of Castilla–La Mancha, Spain, in asso-
ciation with the Computer Human Interaction and
Collaboration Research Group. His research interests
are focused on the fields of new mobile computing,
ubiquitous computing, and m-learning.
Ana I. Molina received the M.Sc. and Ph.D. degrees
in computer science from the Computer Human
Interaction and Collaboration Research Group, Uni-
versity of Castilla–La Mancha, Spain. She is cur-
rently an Associate Professor with the University
of Castilla–La Mancha. She has authored numerous
publications in several national and international
conferences and journals. Her research interests
are, among others, model-based user-interface devel-
opment, human–computer interaction, computer-
supported cooperative work, and collaborative
learning.
40 IEEE REVISTA IBEROAMERICANA DE TECNOLOGIAS DEL APRENDIZAJE, VOL. 11, NO. 1, FEBRUARY 2016

Miguel A. Redondo was born in 1970. He received Reyes Juárez-Ramírez received the master’s
the M.S. degree in computer science from the degree in computational science from the Scien-
University of Granada, Spain, in 1997, and the tific Research and Higher Education Center in
Ph.D. degree from the University of Castilla–La Ensenada, in 2002, and the Ph.D. degree in com-
Mancha, Spain, in 2002. He joined the College putational sciences from the Autonomous Univer-
of Computer Science, University of Castilla–La sity of Baja California. He is currently a Professor
Mancha, in 1999, where he is currently an Associate and Researcher with the Faculty of Chemical Sci-
Professor. His research interests are focused on ences and Engineering, Autonomous University of
the fields of new information technologies applied Baja California. He has two main areas of interest:
to collaborative learning and computer–human software engineering and human–computer interac-
interaction. tion. His current research focuses on adaptive inter-
faces for autistic users, affective aspects in intelligent tutors and videogames,
and affective aspects of users in intelligent spaces.