# Items tagged with statisticsstatistics Tagged Items Feed

### Aggregate Statistics on DataFrames...

July 08 2016 Maple 2016
3
2

Aggregate statistics are calculated by splitting the rows of a DataFrame by each factor in a given column into subsets and computing summary statistics for each of these subsets.

The following is a short example of how the Aggregate command is used to compute aggregate statistics for a DataFrame with housing data:

To begin, we construct a DataFrame with housing data: The first column has number of bedrooms, the second has the area in square feet, the third has price.

bedrooms := <3, 4, 2, 4, 3, 2, 2, 3, 4, 4, 2, 4, 4, 3, 3>:area := <1130, 1123, 1049, 1527, 907, 580, 878, 1075,          1040, 1295, 1100, 995, 908, 853, 856>:price := <114700, 125200, 81600, 127400, 88500, 59500, 96500, 113300,           104400, 136600, 80100, 128000, 115700, 94700, 89400>:HouseSalesData := DataFrame([bedrooms, area, price], columns = [Bedrooms, Area, Price]);

Note that the Bedrooms column has three distinct levels: 2, 3, and 4.

convert(HouseSalesData[Bedrooms], set);

The following returns the mean of all other columns for each distinct level in the column, Bedrooms:

Aggregate(HouseSalesData, Bedrooms);

Adding the columns option controls which columns are returned.

Aggregate(HouseSalesData, Bedrooms, columns = [Price])

Additionally, the tally option returns a tally for each of the levels.

Aggregate(HouseSalesData, Bedrooms, tally)

The function option allows for the specification of any command that can be applied to a DataSeries. For example, the Statistics:-Median command computes the median for each of the levels of Bedrooms.

Aggregate(HouseSalesData, Bedrooms, function = Statistics:-Median);

By default, Aggregate uses the SplitByColumn command to creates a separate sub-DataFrame for every discrete level in the column given by bycolumn.

with(Statistics);
ByRooms := SplitByColumn(HouseSalesData, Bedrooms);

We can create box plots of the price for subgroups of sales defined by number of bedrooms.

BoxPlot( map( (m)->m[Price], ByRooms),              deciles=false,              datasetlabels=["2 bdrms", "3 bdrms", "4 bdrms"],              color=["Red", "Purple", "Blue"]);

I have recorded a short video that walks through this example here: https://youtu.be/e0pqCMyO3ks

### Results of TwoSampleTTest...

June 11 2016
1 5

Hi,

I did some hypothesis testing exercises and I cross checked the result with Maple. I just used following vectors for an unpaired test

a := [88, 89, 92, 90, 90];
b := [92, 90, 91, 89, 91];

I ended up with the following solution:

HFloat(1.5225682336585966)
HFloat(-3.122568233658591)
for a 0.95 confidence interval.

Using

TwoSampleTTest(a, b, 0, confidence = .95, summarize = embed)

and

TwoSampleTTest(a, b, 0, confidence = .975, summarize = embed)

I get following results:

-2.75177 .. 1.15177

-3.13633 .. 1.53633

respectively. I can not explain the discrepancy.

Best regards,

Oliver

PS:

Maple Code in case files won´t be attached.

Unpaired t Test
restart;
Unpaired test-test dataset
a := [88, 89, 92, 90, 90];
b := [92, 90, 91, 89, 91];
The seÂ² estimate is given by:
seÂ²=var(a)+var(b)+2*cov(a*b)=var(a)+var(b)
seÂ²=
sigma[a]^2/Na+sigma[b]^2/Nb;
with Na, Nb being the length of vector a and b respectively.
2                              2
sigma[[88, 89, 92, 90, 90]]    sigma[[92, 90, 91, 89, 91]]
---------------------------- + ----------------------------
Na                             Nb
sigma[a]^2;
and
sigma[b]^2;
are approximated by
S[a]^2;
and
S[b]^2;
2
sigma[[88, 89, 92, 90, 90]]
2
sigma[[92, 90, 91, 89, 91]]
2
S[[88, 89, 92, 90, 90]]
2
S[[92, 90, 91, 89, 91]]
with
S[X]^2;
defined as
S[X]*Â² = (sum(X[i]-(sum(X[j], j = 1 .. N))/N, i = 1 .. N))^2/N;
2
S[X]
2
/      /         N       \\
|      |       -----     ||
|  N   |        \        ||
|----- |         )       ||
| \    |        /    X[j]||
|  )   |       -----     ||
| /    |       j = 1     ||
|----- |X[i] - ----------||
\i = 1 \           N     //
S[X] ï¿ï¾² = ----------------------------
N
with(Statistics);
Sa := Variance(a);
HFloat(2.1999999999999993)
Sb := Variance(b);
HFloat(1.3000000000000003)
Now we are ready to do hypothesis testing (0.95).
We have (with k=min(Na,Nb)=5):
C = mean(a)-mean(b); Deviation := t_(alpha/a, k-1)*se(Sa/k-Sb/k);
c := Mean(a)-Mean(b); deviation := 2.776*sqrt((1/5)*Variance(a)+(1/5)*Variance(b));
HFloat(-0.7999999999999972)
HFloat(2.3225682336585938)
upperlimit := c+deviation; lowerlimit := c-deviation;
HFloat(1.5225682336585966)
HFloat(-3.122568233658591)

Execution of built in student test
TwoSampleTTest(a, b, 0, confidence = .95, summarize = embed);

### An Interactive Application for Exploring Country D...

May 19 2016 Maple 2015
4
0

This is the second of three blog posts about working with data sets in Maple.

In my previous post, I discussed how to use Maple to access a large number of data sets from Quandl, an online data aggregator. In this post, I’ll focus on exploring built-in data sets in Maple.

Data is being generated at an ever increasing rate. New data is generated every minute, adding to an expanding network of online information. Navigating through this information can be daunting. Simply preparing a tabular data set that collects information from several sources is often a difficult and time consuming effort. For example, even though the example in my previous post only required a couple of lines of Maple code to merge 540 different data sets from various sources, the effort to manually search for and select sources for data took significantly more time.

In an attempt to make the process of finding data easier, Maple’s built-in country data set collects information on country-specific variables including financial and economic data, as well as information on country codes, population, area, and more.

The built-in database for Country data can be accessed programmatically by creating a new DataSets Reference:

CountryData := DataSets:-Reference( "builtin", "country" );

This returns a Reference object, which can be further interrogated. There are several commands that are applicable to a DataSets Reference, including the following exports for the Reference object:

exports( CountryData, static );

The list of available countries in this data set is given using the following:

GetElementNames( CountryData );

The available data for each of these countries can be found using:

GetHeaders( CountryData );

There are many different data sets available for country data, 126 different variables to be exact. Similar to Maple’s DataFrame, the columns of information in the built-in data set can be accessed used the labelled name.

For example, the three-letter country codes for each country can be returned using:

CountryData[.., "3 Letter Country Code"];

The three-letter country code for Denmark is:

CountryData["Denmark", "3 Letter Country Code"];

Built-in data can also be queried in a similar manner to DataFrames. For example, to return the countries with a population density less than 3%:

pop_density := CountryData[ .., "Population Density" ]:
pop_density[ Population Density < 3 ];

At this time, Maple’s built-in country data collection contains 126 data sets for 185 countries. When I built the example from my first post, I knew exactly the data sets that I wanted to use and I built a script to collect these into a larger data container. Attempting a similar task using Maple’s built-in data left me with the difficult decision of choosing which data sets to use in my next example.

So rather than choose between these available options, I built a user interface that lets you quickly browse through all of Maple’s collection of built-in data.

Using a couple of tricks that I found in the pages for Programmatic Content Generation, I built the interface pictured above. (I’ll give more details on the method that I used to construct the interface in my next post.)

This interface allows you to select from a list of countries, and visualize up to three variables of the country data with a BubblePlot. Using the preassigned defaults, you can select several countries and then visualize how their overall number of internet users has changed along with their gross domestic product. The BubblePlot visualization also adds a third dimension of information by adjusting the bubble size according to the relative population compared with the other selected countries.

Now you may notice that the list of available data sets is longer than the list of available options in each of the selection boxes. In order to be able to generate BubblePlot animations, I made an arbitrary choice to filter out any of the built-in data sets that were not of type TimeSeries. This is something that could easily be changed in the code. The choice of a BubblePlot could also be updated to be any other type of Statistical visualization with some additional modifications.

You can also interact with it via the MapleCloud: http://maplecloud.maplesoft.com/application.jsp?appId=5743882790764544

I’ll be following up this post with an in-depth post on how I authored the country selector interface using programmatic content generation.

### Why does Mean fail with it?...

May 14 2016
1 8

Below is a custom distribution created based on a function that takes a parameter.

It is possible to create the custom distribution e.g. as D1 and then use it afterwards to find e.g. Mean, but it is not possible to call Mean directly with the creation of the distribution in the call.

Why is that ?

### How to get reverse probability?...

May 13 2016
1 8

When defining a plain standard distributed stochastic variable X, and can find the probability of X <= 0.6 using the Probability function, but how can I get the value for a certain probability, as is done with the fsolve function for example below.

However, the fsolve used to defined Prev above appears to be a bad way to do it, since the Prev function can't for example plot.

Is there some build in way of doing reverse of Probability for a stocastical variable ?

### Maximum likelihood estimation...

May 12 2016
1 5

Hi All

Assume that we have a stochastic model with following density function

and our goal is to estimate unknown parameters namely, alpha, beta, landa, mu and sigma by any available method especially maximum likelihood estimation method.
How can we do it with maple software?

Does the "MaximumLikelihoodEstimate" command can help?

or should i define Maximum Likelihood function first and then differentiate it according to unknown parameters?

Ph.D Candidate

Applied Mathematics Department

### Calculate message's bits...

April 19 2016
0 2

>message:=67A;

67A

>P:=convert(message, bytes);

[54, 55, 65]

>with(Bits):

>bitP1:=Split(P1);

[0, 1, 1, 0, 1, 1]

>bitP2:=Split(P2);

[1, 1, 1, 0, 1, 1]

>bitP3:=Split(P3);

[1, 0, 0, 0, 0, 0, 1]

>with(Statistics):

>b1:=Count(bitP1);

6

>b2:=Count(bitP2);

6

>b3:=Count(bitP3);

7

>totalBits=b1+b2+b3;

19

Hi, how i need to modify my command so when i write any message with any lenght, i can get the totalBits directly..

Thank you~=]]

### RandomVariable vs Distribution in Sample?...

April 13 2016
0 5

Is there a difference between these two?

with(Statistics):

Sample(Normal(0,1),100)

Sample(RandomVariable(Normal(0, 1)), 100)

### Fuzzy c-means clustering...

April 10 2016
2 10

Does Maple have something similar to c-means clusteirng in Matlab?

http://www.mathworks.com/help/fuzzy/fcm.html

How would I go about doing something like this in Maple?

### How to plot critical values?...

March 27 2016
0 3

Is there a way to plot critical values of the Pearson Correlation Coefficient r?  See attached worksheet.  Thanks!

### Error using NonlinearFit function...

March 20 2016
1 6

Howdy all,

I am trying to fit an exponential and logistical model to a set of population data i've been given using the NonlinearFit function in the statistics toolbox. When try to find the fit for the exponential function I get an error saying "SVD of estimated Jacobian could not be computed". Furthermore, when I display the regression over the set of data points all it shows is a horizontal line. I'm not sure how to go about fixing this. My data set is only 17 points and my input function is about as simple as it gets.

When I run the program to solve for logisitcal model I do not get the error but the displayed plot still shows just a horizontal line dispite the function being non-linear.

So far I have...

regE := NonlinearFit(a*exp(b*x),year,population,x)

regLog := NonlinearFit(a/(1+b*exp(-c*x)),year,population,x)

expon := plot((regE), x = 1850..2020):

logi := plot((regLog), x = 1850..2020):

display({data,logi,expon});

I have not tried using optimization yet but I will soon although I'm not sure if it will improve my results since my undertanding is that they both use the same process to estimate the parameters.

Anyways, Thanks for the help in advance!!

EDIT: Here is the data I am using.

year := [1850,1860,1870,1880,1890,1900,1910,1920,1930,1940,1950,1960,1970,1980,1990,2000,2010]:

population :=[4668,9070,17375,27985,37249,63786,115693,186667,359328,528961,806701,1243158,1741912,2049527,2818199,3400578,4092459]:

### How to get back 'Hello'? ...

March 07 2016
1 5

Hi, may I know how I should write the commands after I convert my 'Hello' to 34 and 27 by using the commands below..

Hope someone can help me, thanks a lot..=]] Have a nice day~

message:=’Hello’;

>Hello

plaintext:=convert(message,bytes);

>[72, 101, 108, 108, 111]

P:=numtheory[cfrac](plaintext);

>9418838187/130799212

M1:=numer(P);

>9418838187

M2:=denom(P);

>130799212

with(Bits):

bitM1:=Split(M1);

>[1, 1, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1]

bitM2:=Split(M2);

>[0, 0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1]

with(Statistics):

Count(bitM1);

>34

Count(bitM2);

>27

[72, 101, 108, 108, 111]

### Random points in a for cycle...

February 16 2016
0 4

I have use a ''for cycle'' in order to get a series of points. I would like to save those points in a vector in order to use it for the ''PolynomialFit'' comand. The problem is that the points that I save are sort randomly. How can I take the value of the vector A in the right sequence? in the underline string you can plot the walue of A over t (which is not sorted). I can not use the sort command as I used for t even for A because the points are not increasing.

This is my code:

restart;

Atot := 0:

for ii from 0 by 0.01 to 2 do

PtotFkt := ii->  ii^2 :

Ptot := PtotFkt(ii):

Atot := Atot+0.01*Ptot:

A[ii] := Atot: #Save points in a Table

t[ii] := ii: #Save point in a table

end do;

AV := convert(A, list): #conversion from table to list
nops(AV);  #number of points

timme := convert(t, list): #conversion from table to list
nops(timme); #number of points

with(Statistics); #PolynomialFit

X := Vector(AV, datatype = float);

Y := Vector(sort(timme), datatype = float);

plot(Y, X, style = point, symbol = asterisk, color = blue);

regress := PolynomialFit(10, X, Y, time);

curve1 := plot(regress, time = 0 .. 2);

### Problem with a certain distribution...

February 05 2016
1 7

Hi friends! I have a problem with Random Variable. I don't understand why theoretical Mean differs from sample's Mean

restart; with(Statistics);

r := RandomVariable(NegativeBinomial(3, .1));
Mean((3-1)/(3+r-1));

0.1000000000

S := Sample(r, 10000);

d := map(unapply((3-1)/(3+t-1), t), S);

Mean(d);

0.04703520901756091

But !!

For example if p=0.2 then all is well

### Generate multivariate distribution ...

January 01 2016
0 3

Hi,

I think similar question has been asked by several people, but I did not find a suitable thread. My question is, suppose I have a probablity distirubtion function like

p(x,y) = exp(-alpha (x+y) ) x^2 y^2 / |x-y|  , alpha>0

x,y goes from - \infty to + \infty. This function is normalizable but unbounded, which makes the rejection algorithm a bit difficult(?).

How to generate samping points from this type of probability distribution function?

Thank you very much!

 1 2 3 4 5 6 7 Last Page 1 of 11
﻿