Here is a video of learners playing game in our experimental setup.
The video is of the adaptable SMS based educational game we built this summer. The learners are sent math questions via SMS. When a learner answers a question correctly the difficulty level of consequent questions is raised. If they answer incorrectly then solution to previous answer is sent and questions that follow are of a lesser difficulty level.
Thursday, August 5, 2010
Wednesday, August 4, 2010
Final report
Problem:
Education is billed as the way for people in developing countries to overcome poverty. A number of children never make it to school or are pulled out of one simply because their parents cannot afford it. These unskilled children mostly end up either on the streets or in factories as child laborers.
Given this grim situation regarding literacy, interestingly mobile phones are penetrating at an ever increasing rate with the number of sms sent out in the country topping 35 billion. The cost of mobile phone is low enough for even the low income group comprising mostly of semi-skilled workers to afford mobile phone and its service. Mobile companies also provide very cheap SMS rates to attract customers especially for on-network messaging.
Our idea targets the poor, uneducated masses. It uses sms as a means to educate and consequently help alleviate their current status in life. We, a group of students from LUMS under the guidance of Dr. Umar Saif, thought that it would be an interesting exercise to channel the use of highly available mobile devices to impart education to masses.
The Idea:
Games through SMS service are a fun activity for children and adults alike. Our idea is to use such games to test and teach valuable concepts in maths. The game itself can be self-adaptive, that is, based on responses of the learner, the difficulty level of the questions can be increased or decreased.
The idea was further developed and through brain storming sessions within the group and with Dr. Umar, we decided to have a video lecture followed by SMS games.
Our eventual aim is to take this to the national level by having video lectures aired on TV followed by adaptive game for registered participants.
“Learners” vs. Students:
We would like to call our target audience as learners. This is to emphasize the point that these are people who are eager to learn but do not have the resources, time or access to schools. There is no age or gender restriction neither is their any formal educational setup. To be a learner, you should just be willing to learn, have access to a mobile and some time to watch the televised videos.
Target Learners:
Unlike many other mobile based teaching initiatives, our target learners are people with no formal education beyond primary school. This may include adults and children. Our system primarily caters to people with little formal education. All evaluations found forth are from experiments conducted on similar audiences. It is however worth a mention that our system can very well be used by kids going through formal education for practicing key concepts taught at schools.
According to estimates there are close to 70 million people in Pakistan alone who have no more than high school education. If we target this population then we can attempt to education close to 40% of the population.
Adaptable Education Game for the Masses:
Of of our most challenging tasks was to evaluate a learners progress throughout a game. It is a known fact that different people learn at different rates. In informal distance based education it is very important to keep the learner interested in the game. If a learner gets a number of questions wrong in succession it is highly likely that he will lose interest, so will a player who finds the question too easy.
To solve both these problems, Dr. Umar proposed that we have an adaptable game. In such a game, the difficulty level of questions will adapt according to performance of the learner. If the learner is not able to answer questions then easier questions are asked of the learner. Furthermore, hints in how to solve questions are provided to aid in learning. If the learner is able to answer the questions correctly then he is moved to harder and more challenging questions.
We found this technique very interesting and formed the question accordingly. As you can see from our results this scheme was greatly beneficial for learners in keeping their interest in the game as well as providing them the incentive to progress to harder levels.
Prototype Design:
The goal of our efforts was to create a prototype system which can be expanded for teaching masses while only using existing infrastructure. The focus of this research was to produce a teaching methodology that works. To support the system we set up a VB.net application to process the adaptive behavior supported by an Access database to store questions and results from learners.
The teaching methodology consists of two components; video lectures followed up by adaptable practice games.
Video Lectures:
Following are the video lectures that were shown to the learners.
Language Evaluation:
The language spoken in Pakistan is Urdu. Since most of our learners are expected to have low proficiency in English, we tested our learners in both English and Romanized Urdu. The learners were provided questions in both languages alternatively to evaluate which language is more comprehensive for learners.
Future Vision:
We envision the our system could eventually help solve two big problems. First it can be used to evaluate the effectiveness of the instructor. Secondly we feel that just imparting a lecture is not enough; to truly drive home the concepts our system provides ample opportunity to test the concepts in place.
We developed 3 different games for our learners on multiplication tables, sets and arithmetic. The questions for these games were divided into levels of difficulty as has been described above. The graph given below gives the division of questions in our system.
Figure 1: Division of questions in games.
The arithmetic questions included word problems. Some sample questions are given below:
Level 1:
Q1:
English If you have 4 apples and I give you one more how many apples do you have
Romanized Agar aap kay pass 4 saib hon aur main aap ko 1 aur day doon to aap kay paas kitnay saib hongay.
Level 2:
Q1:
English If you have 4 apples and 2 oranges I give you one more apple how many apples and oranges do you have
Romanized Agar aap kay pass 4 saib aur 2 kinnow hon aur main aap ko 1 aur saib day doon to aap kay paas kitnay saib aur kinnowhongay.
Validation Experiment and Control Group Background:
For this experiment 4 members were selected from the LUMS PEPSI DINING CENTRE. The background of the control group is as follows:
Table-1: Control Group description,
We quizzed the learners before the game to evaluate the positive impact our SMS based game had on the learners.
Results:
The results of the experiment on control group were very positive. There were two important findings which we will discuss here.
1. Effect of lecture
The two charts below (figure 2 and 3) list the responses of the learners before and after lecture. It can be seen that irrespective of the language of question, the learners’ ability to answer increased after the lecture. The increase was close to 30% which is very significant.
Statistics before lecture:
Figure 2: Statistics before the start of experiment.
Statistics after lecture:
Figure 3: Statistics after experiment.
2. Effect of adaptability
Figure 4 shows the rate of correctness with respect to the number of questions. In each time interval we consider 10 questions and plot the number of correct answers against the time interval. As can be seen the learners’ correct response grew slowly at first. Once the learner’s were able to answer correctly for level 1, the level of learners was moved up. This initially resulted in a drop in correctness level but with practice it grew back to 100% correctness. This shows that if learners are provided with adaptable questions then they can learn from these questions and increase their knowledge.
Figure 4: Correctness vs time graph.
Education is billed as the way for people in developing countries to overcome poverty. A number of children never make it to school or are pulled out of one simply because their parents cannot afford it. These unskilled children mostly end up either on the streets or in factories as child laborers.
Given this grim situation regarding literacy, interestingly mobile phones are penetrating at an ever increasing rate with the number of sms sent out in the country topping 35 billion. The cost of mobile phone is low enough for even the low income group comprising mostly of semi-skilled workers to afford mobile phone and its service. Mobile companies also provide very cheap SMS rates to attract customers especially for on-network messaging.
Our idea targets the poor, uneducated masses. It uses sms as a means to educate and consequently help alleviate their current status in life. We, a group of students from LUMS under the guidance of Dr. Umar Saif, thought that it would be an interesting exercise to channel the use of highly available mobile devices to impart education to masses.
The Idea:
Games through SMS service are a fun activity for children and adults alike. Our idea is to use such games to test and teach valuable concepts in maths. The game itself can be self-adaptive, that is, based on responses of the learner, the difficulty level of the questions can be increased or decreased.
The idea was further developed and through brain storming sessions within the group and with Dr. Umar, we decided to have a video lecture followed by SMS games.
Our eventual aim is to take this to the national level by having video lectures aired on TV followed by adaptive game for registered participants.
“Learners” vs. Students:
We would like to call our target audience as learners. This is to emphasize the point that these are people who are eager to learn but do not have the resources, time or access to schools. There is no age or gender restriction neither is their any formal educational setup. To be a learner, you should just be willing to learn, have access to a mobile and some time to watch the televised videos.
Target Learners:
Unlike many other mobile based teaching initiatives, our target learners are people with no formal education beyond primary school. This may include adults and children. Our system primarily caters to people with little formal education. All evaluations found forth are from experiments conducted on similar audiences. It is however worth a mention that our system can very well be used by kids going through formal education for practicing key concepts taught at schools.
According to estimates there are close to 70 million people in Pakistan alone who have no more than high school education. If we target this population then we can attempt to education close to 40% of the population.
Adaptable Education Game for the Masses:
Of of our most challenging tasks was to evaluate a learners progress throughout a game. It is a known fact that different people learn at different rates. In informal distance based education it is very important to keep the learner interested in the game. If a learner gets a number of questions wrong in succession it is highly likely that he will lose interest, so will a player who finds the question too easy.
To solve both these problems, Dr. Umar proposed that we have an adaptable game. In such a game, the difficulty level of questions will adapt according to performance of the learner. If the learner is not able to answer questions then easier questions are asked of the learner. Furthermore, hints in how to solve questions are provided to aid in learning. If the learner is able to answer the questions correctly then he is moved to harder and more challenging questions.
We found this technique very interesting and formed the question accordingly. As you can see from our results this scheme was greatly beneficial for learners in keeping their interest in the game as well as providing them the incentive to progress to harder levels.
Prototype Design:
The goal of our efforts was to create a prototype system which can be expanded for teaching masses while only using existing infrastructure. The focus of this research was to produce a teaching methodology that works. To support the system we set up a VB.net application to process the adaptive behavior supported by an Access database to store questions and results from learners.
The teaching methodology consists of two components; video lectures followed up by adaptable practice games.
Video Lectures:
Following are the video lectures that were shown to the learners.
Language Evaluation:
The language spoken in Pakistan is Urdu. Since most of our learners are expected to have low proficiency in English, we tested our learners in both English and Romanized Urdu. The learners were provided questions in both languages alternatively to evaluate which language is more comprehensive for learners.
Future Vision:
We envision the our system could eventually help solve two big problems. First it can be used to evaluate the effectiveness of the instructor. Secondly we feel that just imparting a lecture is not enough; to truly drive home the concepts our system provides ample opportunity to test the concepts in place.
We developed 3 different games for our learners on multiplication tables, sets and arithmetic. The questions for these games were divided into levels of difficulty as has been described above. The graph given below gives the division of questions in our system.
Figure 1: Division of questions in games.
The arithmetic questions included word problems. Some sample questions are given below:
Level 1:
Q1:
English If you have 4 apples and I give you one more how many apples do you have
Romanized Agar aap kay pass 4 saib hon aur main aap ko 1 aur day doon to aap kay paas kitnay saib hongay.
Level 2:
Q1:
English If you have 4 apples and 2 oranges I give you one more apple how many apples and oranges do you have
Romanized Agar aap kay pass 4 saib aur 2 kinnow hon aur main aap ko 1 aur saib day doon to aap kay paas kitnay saib aur kinnowhongay.
Validation Experiment and Control Group Background:
For this experiment 4 members were selected from the LUMS PEPSI DINING CENTRE. The background of the control group is as follows:
Table-1: Control Group description,
We quizzed the learners before the game to evaluate the positive impact our SMS based game had on the learners.
Results:
The results of the experiment on control group were very positive. There were two important findings which we will discuss here.
1. Effect of lecture
The two charts below (figure 2 and 3) list the responses of the learners before and after lecture. It can be seen that irrespective of the language of question, the learners’ ability to answer increased after the lecture. The increase was close to 30% which is very significant.
Statistics before lecture:
Figure 2: Statistics before the start of experiment.
Statistics after lecture:
Figure 3: Statistics after experiment.
2. Effect of adaptability
Figure 4 shows the rate of correctness with respect to the number of questions. In each time interval we consider 10 questions and plot the number of correct answers against the time interval. As can be seen the learners’ correct response grew slowly at first. Once the learner’s were able to answer correctly for level 1, the level of learners was moved up. This initially resulted in a drop in correctness level but with practice it grew back to 100% correctness. This shows that if learners are provided with adaptable questions then they can learn from these questions and increase their knowledge.
Figure 4: Correctness vs time graph.
Tuesday, July 27, 2010
The Conclusion
During the summer, we worked in two groups to create an SMS based game that can help in teaching masses. Both the groups came up with fascinating ideas, but interestingly we had similar findings. You can examine the results of our various experiments in the previous entries. From these results we have drawn following conclusions.
1. From our statistics, we have found that a lecture prior to the game is extremely beneficial (upto 33% in one control group). This implies that, if we tie up the game with lectures on public media like TV broadcast, followed by SMS questions then this system can become an extremely effective mass education system.
2. Participants were more comfortable with multiple choice questions. This is validated by both the statistics (more correct answers through MCQ) and interviews (see below).
3. Our language experimentation was different with different conclusions. First group provided every question both in English and in Romanized Urdu. Their conclusion was that participants found questions written in both the languages as helpful at the same time. Other group provided questions alternately in English and in Romanized Urdu. Their conclusion was that results are marginally (10%.) better for Romanized Urdu.
As a final remark, we strongly claim that this SMS based teaching system can greatly aid in teaching masses. Through this system people who have left education due to some reason can have a chance to continue their education and learn new concepts that otherwise may be not possible.
1. From our statistics, we have found that a lecture prior to the game is extremely beneficial (upto 33% in one control group). This implies that, if we tie up the game with lectures on public media like TV broadcast, followed by SMS questions then this system can become an extremely effective mass education system.
2. Participants were more comfortable with multiple choice questions. This is validated by both the statistics (more correct answers through MCQ) and interviews (see below).
3. Our language experimentation was different with different conclusions. First group provided every question both in English and in Romanized Urdu. Their conclusion was that participants found questions written in both the languages as helpful at the same time. Other group provided questions alternately in English and in Romanized Urdu. Their conclusion was that results are marginally (10%.) better for Romanized Urdu.
As a final remark, we strongly claim that this SMS based teaching system can greatly aid in teaching masses. Through this system people who have left education due to some reason can have a chance to continue their education and learn new concepts that otherwise may be not possible.
Results!
After a series of experiments, here are the concluding results of our project.
First we would like to talk about the response of users over time. The graph below shows the average response of users in games through SMS. As you can see, as the participant started to answer the questions, their responses got better. The dip at the 19th question is when two participants changed level and faced difficulty in the an advanced level.
Before the lecture
The second result shows the response to the lecture given to the participants. We observed a 33% increase in their success after they were given a lecture on the material as shown in graphs below.
After the lecture
First we would like to talk about the response of users over time. The graph below shows the average response of users in games through SMS. As you can see, as the participant started to answer the questions, their responses got better. The dip at the 19th question is when two participants changed level and faced difficulty in the an advanced level.
Before the lectureThe second result shows the response to the lecture given to the participants. We observed a 33% increase in their success after they were given a lecture on the material as shown in graphs below.
After the lecture
Thursday, July 22, 2010
Subscribe to:
Posts (Atom)